About the teamDialpad’s QA Analysts work closely within our AI teams to ensure that our customers can reap the benefits of AI without a high risk of unexpected outcomes or problematic outputs. In this role, you’ll combine test set creation and curation with automated, performance, and manual testing to ensure the best possible performance, security, and user experience for our AI features.Dialpad’s AI Data team plays an essential role in ensuring that we are handling our customer data with care, creating representative datasets for testing, and verifying the outcomes of models are what we expect in output and performance. This team collaborates closely with the automatic speech recognition, natural language processing, data analysis, and AI Engineering teams to constantly improve our AI models as well as create new models that drive all of Dialpad’s AI features. Your roleAs a QA Analyst, you’ll own testing all of our new AI models from speech recognition to various ML and LLM models. You’ll work closely with our data annotation team, AI scientists and engineers to determine what tests need to be run, create test plans, design and create test sets, and execute on those test plans for new features or model architectures. You’ll also help run existing tests on our regular cadence releases that ensure we have no serious regressions while keeping our models up to date for our customers. This position reports to our Manager, Data Annotation and Testing and has the opportunity to be based in our Kitchener office with the ability to work remotely in the area.What you’ll do Given an AI feature, be able to determine what tests should be run and what data will be required to ensure quality.Create test plans and scripts for all levels of testing from smoke tests to unit tests to performance and security and report issues in JIRA.Perform automated and manual tests of new and existing features to ensure they are functioning as expected.Execute and coordinate red teaming, smoke testing, system integration testing, regression testing, security testing, performance testing, and end-to-end testing in support of test plans.Create and design tests and metrics to measure both qualitative and quantitative aspects of ML models.Be able to communicate with both technical and non-technical audiences to report the results of your testing.Able to collaborate with cross-functional teams(ASR, NLP, Analytics) to align on testing objectives and requirements.Skills you’ll bring3+ years of experience as a QA Analyst or equivalent role testing ML models.Experience with Python, Docker, SQL, Google Cloud and Data Visualization tools.Familiarity with version control tools like Git for collaborative projects.Knowledge about machine learning models (ASR and NLP).Experience with automated, performance, and manual testing.Flexible with regard to changing priorities & business needs.Strong analytical and problem-solving skills.Ability to quickly learn and attention to details.Excellent written and verbal communication skills.If you speak or read other natural languages (e.g., Spanish, French) that would be an added bonus.Dialpad benefits and perksBenefits, time-off, and wellnessAn apple a day keeps the doctor away—and it doesn’t hurt that we offer flexible time off and great options for medical, dental, and vision plans for all employees. Along with that, employees also receive a monthly stipend to help cover your cell phone bill, home internet bill, and we reimburse for gym membership costs, a variety of wellness events, and more!Professional DevelopmentDialpad offers reimbursement for expenses related to professional development, up to an annual limit per calendar year.
