Behind the Scenes with Prompts
Similar to others looking to leverage GAI, we were learning how to author large language model (LLM) prompts and quickly discovered that prompt workflows are quite involved, each requiring a few fundamental building blocks. Each new iteration of a prompt required a few hours of inference to generate sample responses, which would get collected into huge spreadsheets and distributed to a team of editors. These editors would review the prompt outputs from the previous day and score the results, providing a signal for the next prompt iteration. We needed to streamline this workflow, so we built a versioned prompt templating system, single or multi-step prompts, and human and automated prompt response evaluation that incorporated both content quality and scoring.
Prototyping with Tape – Rip, Replace, Repeat
Our ‘hack-track’ was a team of three engineers on a mission to create new variants of Collaborative Articles so that we could explore different avenues. They were on a daily mission to create new variants of the member experience. The mocked-up prototypes in code allowed internal teams to play with changes, provide feedback, and keep the member interface iterations rolling. For us, it was important to keep that level of flexibility and abstraction in the prototyping phase. It set the teams up to explore and truly learn from this new build experience.
Connecting the Dots
After extensive prototyping, it was time to connect the dots. We scaled up topic generation, sent starter articles for editorial review, and slotted approved articles into publishing queues. Then came distribution, which involved connecting millions of experts with members needing insights. We planned to ensure these expert voices reached members who’d use these insights where they were, finding them through search engines or engaging with them on LinkedIn in their feeds, InMails, and notifications. Once all the pieces aligned, we were ready to share Collaborative Articles with our broader LinkedIn community.
Signals of Mastery
An integral part of this infrastructure was recognizing real experts, which was harder than it seemed. Though we’re equipped with many direct and indirect signals to gauge a member’s skill proficiency, there’s also a lot of noise. Some of these signals have low coverage, while others have low precision. Ultimately, signals that worked reasonably were a combination of explicit skills – skills on profiles, skill endorsements from others, recent job titles – and implicit skills, which are inferred based on recent hires for job postings or a member’s self-evaluation during job applications.
Additionally, we added insights such as the likelihood of the member adding original thoughts based on their past content shared on LinkedIn. This gave us a reasonable starting point for expert identification. Once experts started contributing their thoughts, we fine-tuned this process and assessed how engaging and valuable expert opinions were to the community. This is where the magic happened – turning a combination of raw signals into actionable insights where we can better match experts with articles where they can add their perspectives, answers, and insights.
Supporting Trust While Surfacing the Gems
At LinkedIn, trust forms the bedrock of everything we build. For Collaborative Articles, where we invite multiple experts to contribute to a single question, our defenses had to be smart not to stifle diverse viewpoints and healthy debates. Yet, they must identify and quickly remove unwanted, policy-violating content. Our trust classifiers actively identify and filter out unsafe, harmful, and unprofessional content that doesn’t align with our platform policies. Our reactive defenses also empower members to swiftly report problematic content, enabling us to take quick action to safeguard our community. We enforce strict penalties for repeat offenders, revoking contribution access to any future articles. This comprehensive approach ensures that we invite and encourage diverse viewpoints in a safe, professional environment.
Six Months In: From Launch to Growing Momentum
It’s been just over six months since the launch of collaborative articles. We’ve celebrated our one-millionth expert contribution, and it’s clear that more and more members are turning to expert answers as their guidebook for work-related challenges. In the past month alone, we’ve witnessed a 74% surge in the number of articles read by our members. Collaborative Articles are also swiftly becoming one of the fastest-growing traffic drivers to LinkedIn. As engineers, there’s no greater thrill than providing people with a powerful platform for expression on a massive scale.
Reflecting on this journey, we recognize the many factors contributing to our success. However, with their boundless enthusiasm and exceptional talent, our dream team reigns supreme at the top of that list.
