Psychometrician
Altus Assessments, Canada
Job Description
Our Purpose:
We are here to create a world served by exceptional professionals. Our products (Casper, Snapshot, and Duet) help higher education institutions to look beyond book smarts, seeing a clearer, more holistic view of applicants. It’s used by 360+ academic partners in Canada, US, Australia (and growing!) with 150,000+ applicants taking the test each year, and covers 90% of all US medical school applicants. We also host the Admissions Summit, an annual 2-day conference, and power the Alo Grant, an annual $100,000 research fund.
The Role:
Are you interested in engaging in diverse and innovative research in the field of assessment and admissions? We are looking for a psychometrician to ensure defensibility and fairness of our assessment by conducting rigorous research with existing and novel data from our applicants. You will have the opportunity to make a meaningful impact on academic admissions by helping our partners select the best applicants for their professional programs. You will work closely with our content and rating teams and engage in all stages of research including study design, instrument evaluation, data cleaning and analysis, report and publication writing, and presentations. Specifically, you will be responsible for: providing expert guidance in statistical methods for education and assessment, using R and Looker to derive insights about the reliability and validity of the Casper test, and using these insights to make tangible improvements to our processes and the test as a whole.
This is a full-time role reporting to the Director of Research within the Internal Research Team, ideally located in Ontario but open to other parts of Canada (and comfortable working from home for at least the next 9-12 months while we maintain our CV-19 precautions).
Within 1 month, you will:
- Onboard and get to know the team! We will work hard to make sure you feel supported and well-prepared to take on your new role
- Observe and contribute in our team meetings. You’ll get a clearer picture of how things work on the research team and with our core product Casper, while also giving input about how things can get even better (fresh eyes are key to continuous improvement)
- Learn about the other teams at Altus. Part of combatting siloing at Altus is taking the time to get to know your colleagues across the company. Book 1v1s, coffee chats, sit in on meetings - do whatever you need to do to meet the AMAZING people at Altus and understand how all of the pieces of the puzzle work together.
- Get to know our test, inside and out. Spend some time taking the practice test, completing rater training, learning about our content, exploring the data, asking a ton of questions to all of the relevant teams’ experts, and generally learning as much as you can about the test you’ll be working so closely with!
Within 3 months, you will:
- Have a good understanding of the inner workings of the Casper test and what makes it unique, and the types of analyses we can conduct to validate the assessment - and start to generate novel insights of your own!
- Be knowledgeable about the ongoing research projects at Altus related to the Casper test across our Research, Operations, Product and Data Science teams.
- Start to think about your own list of research projects- uncover new research priorities and make a case for why they should be at the top of our list
- Begin to own the reliability and validity of the Casper test- know what’s working well, what’s not, and most importantly- why. Be able to speak to the psychometric performance of the test as well as the elements that need improvement, and start to come up with a plan to get there
- Develop close working relationships with all of these teams, and know who to connect with to make good cross-functional decisions.
Within 12 months, you will:
- You are enthusiastically engaged in analyses of our vast data, and in creating plans that directly contribute to new tools that serve our purpose of creating a world of better professionals.
- Your work inspires those around you, and you apply analyses to ensure that the psychometric properties of our tools are tested in an ongoing fashion, and you are eager to provide meaningful insights into our tools.
- The team is buzzing with excitement about being involved in the plans you helped create to improve existing products, and to create new ones.
- Your findings make a difference! They have real world applications and enhance the Altus goal of promoting excellence in professionals.
- You use powerful data visualization techniques that capture the imagination of your audience.
- You published your work in journal articles and present at conferences enhancing recognition of Altus as a