I’m working as a Sr. Machine Learning DA in one of the biggest technology and leading AI organizations in the world, AMAZON. I am in the department of Alexa (yes, you heard right) and standing in the role of an SME for the Indian region in my project. We primarily deal with big data pre-processing, data cleansing, and wrangling operations to perform text and sentiment analysis, evaluate device performance and provide measures to improve model accuracy and model performance.
As a part of my daily work routine, I have to review the errors the team made for the past day and copy-paste this content into a quip sheet (similar to an excel workbook) to be used by my managers or by me to analyze the data and find trends; it also acts as a repository for associates to look back for reference.
This process, however, was being manually done every day, and each associate was given 30 mins to do the same. That means a team of 9 members takes 4.5 hrs daily for this tedious task, so the problem with manual data entry is that it’s time-consuming, and there’s always room for data tampering. Since the team uses a common shared file, the possible risk of losing data and difficulty with maintaining such a repository is high.
So, I wanted to automate this process. I approached my manager with this problem statement and proposed a solution to automate the process after getting the required permissions.
I did my research on the frameworks to achieve this, and I started building a DB model kind of setup using Python, which extracts information from the Error repository and feeds it back to the Quip sheet in an organized fashion.
I used Pandas and NumPy libraries, some user-defined functions, and frameworks to build the model. The model extracts individual D.A.’s information alphabetically, fits the raw data in organized rows and columns, calculates error scores for each record, and finally exports it as a quip file.
On average, the model was able to save 23 mins of production time for each D.A. meaning 207 minutes saved on a daily basis which is 76.67% more time-efficient than the manual process before that.
The project earned me the High-Five award in the Rewards and Recognition ceremony that took place in Dec 2021.
If you also wish to work on such projects and implement them at work, you can take up the PGP Data Science and Business Analytics Course and upskill today.
