New Relic helps nonprofit CareerVillage enter the AI era

At CareerVillage.org, we connect over 135,000 professionals to support millions of students of all ages in career entry and advancement. We are now the go-to source for real-world information on education and career advancement advice. We use a range of technologies to manage our product, like Django and React, with multiple environments for staging and production. We've got databases of various kinds: mostly Postgres, in addition to Redis, Solar, and Celery, and data pipelines to populate our warehouse. We also have some machine learning services that help with things like content moderation and content tagging.

We first started using New Relic to help us with server monitoring. Twelve years ago, when we went into schools with hundreds of students and teachers, they wouldn't be able to log in at the same time. We used to ask students to log in alphabetically. First, all students with names starting with “a” logged in. Then, after a minute, ask students beginning with “b” logged in, and so on. New Relic helped us identify what the server and load balancing issues were, so we could grow to enable 700 students to log in concurrently.  It helped us introduce the right technologies, like load balancing, when there were gaps in our observability that improved the user experience from an infrastructure point of view.

Now we monitor our dev, staging, and production environments. We also have a separate load testing environment that we monitor, in addition to our uptime. From there we look at how we can reduce bottlenecks, such as concurrent writing to our databases. We also track mean time to recovery (MTTR), and empty cache to page load time, which is essential when so much of our traffic comes from search engines. Using New Relic, we were able to get this load time down from 300 milliseconds to under 100 milliseconds.

Introducing AI

Over the past year, we have invested in building an AI career coach. We need to be a nimble organization, especially on the engineering side. With New Relic, we have been able to focus new engineers on our AI features, and not on our infrastructure monitoring. There are a lot of technologies involved: a frontend React application, a backend Django application running in the cloud, LangChain as an AI orchestrator, agent management libraries, and a combination of third-party large language models (LLMs) and our own trained and self-hosted LLM,  retrieval augmented generation tools, and Pinecone DB for the vector database that supports the retrieval.

Our AI career coach represents a shift in our business. The goal as always is to help young people, but now we can also support adult learners. We're able to serve adults who are looking to change their job or help them understand what they might need to grow their careers. The goal of the AI career coach is not only to answer questions from our crowdsourced data, but do exercises like mock interviews, review a job description against your qualifications, or have a conversation about your networking abilities. These are less like question-answer features and more about building a learning experience. We're already deploying it in schools, nonprofits, and workforce development training programs. The AI career coach needs to be reliable, consistent, and predictable. There’s not much tolerance for bugs or problems.

Our new AI tool connects to New Relic as well, where we monitor its uptime, performance, and response times. It has been because New Relic works so well that it has made entry into these new domains possible: our engineering team is growing from three core engineers to seven, but our new engineers are not focused on monitoring. It's keeping costs down by allowing our developers to focus on the user-facing product features that matter.

As a small team, New Relic helps us build out monitoring for our crowdsourced platform. It has grown with us to support our phenomenal student and user growth and allows our engineering team to focus on building out new services and features. New Relic is our core support for observability: it is our team member focused on supporting our DevOps functions. We don't want to hire more engineers to focus just on observability, that is the role of New Relic in our organization.