{"id":17170,"date":"2025-06-30T21:03:30","date_gmt":"2025-06-30T21:03:30","guid":{"rendered":"https:\/\/www.comet.com\/site\/?page_id=17170"},"modified":"2025-11-17T20:58:16","modified_gmt":"2025-11-17T20:58:16","slug":"zencoder-ai-code-generator","status":"publish","type":"page","link":"https:\/\/www.comet.com\/site\/customers\/zencoder-ai-code-generator\/","title":{"rendered":"How Opik Helps Zencoder Build &#038; Test Fully Agentic Software Pipelines"},"content":{"rendered":"\n<p>The team behind Zencoder is building products that advance software engineering through intelligent automation. Zencoder is an AI-powered coding agent that enhances user productivity by automating tasks such as real-time code repair, unit test generation, documentation, and terminal operations, all designed to streamline the software development lifecycle.<\/p>\n\n\n\n<p>We sat down with Dmitrii Krasnov, Engineering Manager at Zencoder, who leads the research team and acts as product owner for Zencoder\u2019s key agent features, to discuss how Opik\u2019s <a href=\"https:\/\/www.comet.com\/site\/blog\/llm-evaluation-guide\/\" target=\"_blank\" rel=\"noreferrer noopener\">LLM evaluation<\/a> platform has provided a centralized way to for team members to evaluate LLM outputs, streamline debugging, and align both technical and non-technical teams across workflows.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-scaling-smarter-agents-demands-smarter-infrastructure\">Scaling Smarter Agents Demands Smarter Infrastructure<\/h2>\n\n\n\n<p>As more companies shifted from using simple AI assistants to fully integrated agent pipelines, they needed to ensure that Zencoder could scale with them. Zencoder orchestrates complex tasks, such as coding, testing, planning, managing JIRA tickets, integrating documentation, and enforcing code style.<\/p>\n\n\n\n<p>Zencoder has developed AI agents that developers can use in their IDE and autonomous agents that run within the DevOps pipeline, allowing users to assign a JIRA ticket and have the agent resolve the ticket from start to finish. For this to be possible, they needed to have a reliable and observable infrastructure in place to support this development.<\/p>\n\n\n\n<div class=\"wp-block-group is-layout-constrained wp-block-group-is-layout-constrained\">\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>&#8220;LLMs are black boxes. We don\u2019t know what is going on inside them. We needed a solution that allowed us to see how our models behaved, and have the ability to understand what went wrong, and share that with the team to debug and iterate faster.&#8221;<\/p>\n<cite>Dmitrii Krasnov, Engineering Manager at Zencoder<\/cite><\/blockquote>\n<\/div>\n\n\n\n<p><strong>Before adopting Opik, Dimitrii\u2019s team encountered several pain points common with building LLM systems:<\/strong><\/p>\n\n\n\n<div class=\"wp-block-group is-layout-constrained wp-block-group-is-layout-constrained\">\n<ul class=\"wp-block-list\">\n<li>Limited transparency into how models behaved under different conditions<\/li>\n\n\n\n<li>Slow iteration cycles caused by manual, time-intensive debugging<\/li>\n\n\n\n<li>Fragmented collaboration between engineering, research, and customer-facing teams<\/li>\n<\/ul>\n<\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-leveling-up-llm-development-with-centralized-evaluation\">Leveling Up LLM Development With Centralized Evaluation<\/h2>\n\n\n\n<p>The team at Zencoder was already using Comet\u2019s <a href=\"https:\/\/www.comet.com\/site\/products\/ml-experiment-tracking\/\">ML experiment management<\/a> tooling to support their ML workflows, but as Zencoder\u2019s LLM agents began handling more complex tasks, the team needed deeper visibility into how models were reasoning and where failures were occurring. That&#8217;s where Opik came in, offering a centralized platform to trace agent behavior, compare model outputs, and tighten the feedback loop between research and product teams.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/06\/ai-code-generator-obervability-e1751395831491.png\" alt=\"dashboard screenshot showing a logged trace for an ai code generator\"\/><\/figure>\n\n\n\n<p>Opik\u2019s tracing and annotation UI makes it easy for subject matter experts outside the application development team to review LLM outputs and provide feedback to Dmitrii\u2019s team directly inside the platform. <\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\u201cOur first priority was to have someone with domain knowledge go through traces and figure out why the model wasn&#8217;t acting as expected. That\u2019s where observability becomes essential.\u201d<\/p>\n<\/blockquote>\n\n\n\n<p>With Opik, researchers can track subtle variations between prompts, verify metadata across experiments, and quickly identify regressions. It also enables less technical team members to engage directly with model behavior, thereby strengthening collaboration and accelerating decision-making across the board.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-the-results-of-building-with-better-feedback-loops\">The Results of Building With Better Feedback Loops<\/h2>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>&#8220;If you\u2019re not iterating quickly, you\u2019re falling behind. Comet\u2019s Opik helps us move faster without losing quality.&#8221;<\/p>\n<\/blockquote>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-faster-development-cycles\">Faster Development Cycles<\/h3>\n\n\n\n<p>Adopting Opik has helped scale Zencoder more efficiently without compromising on quality or visibility. Iteration cycles are now faster than ever, helping the team stay nimble to experiment and ship new features with confidence.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-improved-research-productivity\">Improved Research Productivity<\/h3>\n\n\n\n<p>Opik was built to scale and supports the team\u2019s high volume of experimentation, as they are currently running thousands of experiments daily. <\/p>\n\n\n\n<div class=\"wp-block-group is-layout-constrained wp-block-group-is-layout-constrained\">\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\u201cWith so many researchers and experiments, it\u2019s easy to miss small changes. Being able to check that everything\u2019s aligned and troubleshoot unexpected behaviors is crucial.\u201d<\/p>\n<\/blockquote>\n<\/div>\n\n\n\n<p>The team now has the ability within Opik to quickly search across traces, compare outputs, and inspect metadata, which is crucial for validating that the prompts and models are performing as needed, making it easier to catch minor issues or regressions before they snowball.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"520\" src=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/06\/ai-code-generator-evaluation-1024x520.png\" alt=\"dashboard screenshot showing comparison of llm outputs for an ai code generator\" class=\"wp-image-17196\" srcset=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/06\/ai-code-generator-evaluation-1024x520.png 1024w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/06\/ai-code-generator-evaluation-300x152.png 300w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/06\/ai-code-generator-evaluation-768x390.png 768w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/06\/ai-code-generator-evaluation-1536x780.png 1536w, https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/06\/ai-code-generator-evaluation-2048x1040.png 2048w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-streamlined-cross-functional-collaboration\">Streamlined Cross-Functional Collaboration<\/h3>\n\n\n\n<p>Opik\u2019s shared workspace has improved collaboration across roles. All team members, regardless of their technical background, now have visibility into traces and can follow how models are behaving, and quickly loop in the correct people once an issue is identified.<\/p>\n\n\n\n<div class=\"wp-block-group is-layout-constrained wp-block-group-is-layout-constrained\">\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>&#8220;It\u2019s easier for non-technical team members to understand what\u2019s going on. Just having one centralized tool where everyone can check and understand the context has been a huge win.&#8221;<\/p>\n<\/blockquote>\n<\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-opik-adds-critical-observability-and-unit-testing-for-scalable-agents\">Opik Adds Critical Observability and Unit Testing for Scalable Agents<\/h2>\n\n\n\n<p>Opik has become an essential part of Zencoder\u2019s developers&#8217; workflow, assisting the team in building, testing, and scaling Zencoder&#8217;s capabilities. It enables faster iteration, more efficient experimentation, and improved collaboration across technical and non-technical teams. As Zencoder continues to build smarter agents, Opik ensures they have the necessary LLM eval infrastructure to support their efforts.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Unlock Reliable Performance for Your Complex GenAI Applications<\/h3>\n\n\n\n<p>Opik brings clarity to complex LLM development, whether you&#8217;re working on scaling and validating AI code assistants or another type of complex agentic system. <a href=\"https:\/\/www.comet.com\/site\/about-us\/contact-us\/\">Contact us<\/a> today to learn how Opik provides the observability and iteration layer you need to ship trustworthy AI systems to a massive user base, with <strong>personalized, technical attention to your team&#8217;s specific needs and goals: <\/strong><\/p>\n\n\n\n<div class=\"wp-block-buttons is-content-justification-center is-layout-flex wp-container-core-buttons-is-layout-749adb30 wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link has-text-align-center wp-element-button\" href=\"https:\/\/www.comet.com\/site\/about-us\/contact-us\/\">Contact Us<\/a><\/div>\n<\/div>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The team behind Zencoder is building products that advance software engineering through intelligent automation. Zencoder is an AI-powered coding agent that enhances user productivity by automating tasks such as real-time code repair, unit test generation, documentation, and terminal operations, all designed to streamline the software development lifecycle. We sat down with Dmitrii Krasnov, Engineering Manager [&hellip;]<\/p>\n","protected":false},"author":140,"featured_media":18116,"parent":488,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"case-study","meta":{"customer_name":"Zencoder","customer_description":"Zencoder builds AI-powered coding agents to support developers throughout the software development process. Their coding agents automate tasks like code generation, bug fixing, testing, and documentation, enabling users to build and ship software faster.","customer_industry":"Technology - AI Coding Assistants","customer_technologies":"Opik","customer_logo":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/10\/zencoder-logo-light.svg","footnotes":""},"coauthors":[127],"class_list":["post-17170","page","type-page","status-publish","has-post-thumbnail","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.9 (Yoast SEO v25.9) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>How Zencoder Scales Smarter AI Coding Agents with Opik<\/title>\n<meta name=\"description\" content=\"Zencoder is building AI code gen agents that can actually create and resolve their own Jira tickets. Opik tracks and evaluates those complex LLM calls under the hood.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.comet.com\/site\/customers\/zencoder-ai-code-generator\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How Opik Helps Zencoder Build &amp; Test Fully Agentic Software Pipelines\" \/>\n<meta property=\"og:description\" content=\"Zencoder is building AI code gen agents that can actually create and resolve their own Jira tickets. Opik tracks and evaluates those complex LLM calls under the hood.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.comet.com\/site\/customers\/zencoder-ai-code-generator\/\" \/>\n<meta property=\"og:site_name\" content=\"Comet\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cometdotml\" \/>\n<meta property=\"article:modified_time\" content=\"2025-11-17T20:58:16+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/10\/Case-study-Zencoder.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"1842\" \/>\n\t<meta property=\"og:image:height\" content=\"650\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@Cometml\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n\t<meta name=\"twitter:label2\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data2\" content=\"Caroline Borders\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"How Zencoder Scales Smarter AI Coding Agents with Opik","description":"Zencoder is building AI code gen agents that can actually create and resolve their own Jira tickets. Opik tracks and evaluates those complex LLM calls under the hood.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.comet.com\/site\/customers\/zencoder-ai-code-generator\/","og_locale":"en_US","og_type":"article","og_title":"How Opik Helps Zencoder Build & Test Fully Agentic Software Pipelines","og_description":"Zencoder is building AI code gen agents that can actually create and resolve their own Jira tickets. Opik tracks and evaluates those complex LLM calls under the hood.","og_url":"https:\/\/www.comet.com\/site\/customers\/zencoder-ai-code-generator\/","og_site_name":"Comet","article_publisher":"https:\/\/www.facebook.com\/cometdotml","article_modified_time":"2025-11-17T20:58:16+00:00","og_image":[{"width":1842,"height":650,"url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/10\/Case-study-Zencoder.webp","type":"image\/webp"}],"twitter_card":"summary_large_image","twitter_site":"@Cometml","twitter_misc":{"Est. reading time":"4 minutes","Written by":"Caroline Borders"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.comet.com\/site\/customers\/zencoder-ai-code-generator\/","url":"https:\/\/www.comet.com\/site\/customers\/zencoder-ai-code-generator\/","name":"How Zencoder Scales Smarter AI Coding Agents with Opik","isPartOf":{"@id":"https:\/\/www.comet.com\/site\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.comet.com\/site\/customers\/zencoder-ai-code-generator\/#primaryimage"},"image":{"@id":"https:\/\/www.comet.com\/site\/customers\/zencoder-ai-code-generator\/#primaryimage"},"thumbnailUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/10\/Case-study-Zencoder.webp","datePublished":"2025-06-30T21:03:30+00:00","dateModified":"2025-11-17T20:58:16+00:00","description":"Zencoder is building AI code gen agents that can actually create and resolve their own Jira tickets. Opik tracks and evaluates those complex LLM calls under the hood.","breadcrumb":{"@id":"https:\/\/www.comet.com\/site\/customers\/zencoder-ai-code-generator\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.comet.com\/site\/customers\/zencoder-ai-code-generator\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/customers\/zencoder-ai-code-generator\/#primaryimage","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/10\/Case-study-Zencoder.webp","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/10\/Case-study-Zencoder.webp","width":1842,"height":650,"caption":"Opik x Zencoder"},{"@type":"BreadcrumbList","@id":"https:\/\/www.comet.com\/site\/customers\/zencoder-ai-code-generator\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.comet.com\/site\/"},{"@type":"ListItem","position":2,"name":"Customers","item":"https:\/\/www.comet.com\/site\/customers\/"},{"@type":"ListItem","position":3,"name":"How Opik Helps Zencoder Build &#038; Test Fully Agentic Software Pipelines"}]},{"@type":"WebSite","@id":"https:\/\/www.comet.com\/site\/#website","url":"https:\/\/www.comet.com\/site\/","name":"Comet","description":"Build Better Models Faster","publisher":{"@id":"https:\/\/www.comet.com\/site\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.comet.com\/site\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.comet.com\/site\/#organization","name":"Comet ML, Inc.","alternateName":"Comet","url":"https:\/\/www.comet.com\/site\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/","url":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","contentUrl":"https:\/\/www.comet.com\/site\/wp-content\/uploads\/2025\/01\/logo_comet_square.png","width":310,"height":310,"caption":"Comet ML, Inc."},"image":{"@id":"https:\/\/www.comet.com\/site\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/cometdotml","https:\/\/x.com\/Cometml","https:\/\/www.youtube.com\/channel\/UCmN63HKvfXSCS-UwVwmK8Hw"]}]}},"_links":{"self":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/pages\/17170","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/users\/140"}],"replies":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/comments?post=17170"}],"version-history":[{"count":3,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/pages\/17170\/revisions"}],"predecessor-version":[{"id":18477,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/pages\/17170\/revisions\/18477"}],"up":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/pages\/488"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media\/18116"}],"wp:attachment":[{"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/media?parent=17170"}],"wp:term":[{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.comet.com\/site\/wp-json\/wp\/v2\/coauthors?post=17170"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}