03: Meeting growth challenges; evolving scientific software deployment

In our first article, we laid out why we’re writing this series—to explore the challenges of modernizing scientific software infrastructure and to share the solutions we’re adopting. Scientific research depends on efficiency and reliability, yet many organizations still rely on outdated, manual deployment processes. At Elixir Software, we’re evolving our approach to keep pace with growing demands.

If you’ve ever built software for scientific research, you already know one universal truth: what starts as a simple, elegant solution eventually turns into a sprawling, interconnected system. In the early days of Elixir Software, our deployment needs were straightforward. We had a single product, a small number of deployments, and a manageable infrastructure.

Fast forward to today and things look very different, as the number of iTraX deployments has rapidly expanded. Our product suite has evolved from a single workflow configuration of iTraX to over six key industry-standard workflows, each with its own complexities. Meanwhile, our cloud team remains lean, which, while efficient, also requires careful strategy and automation to scale effectively.

This is the natural progression of a growing scientific SaaS business, and we are actively adapting to meet these new demands.

The Challenges of Growth

As we scale, we face new technical and operational considerations. What worked in the past needs to evolve to meet increasing demands. Here’s what we’re adapting to:

• Scaling deployments efficiently
  With deployments increasing exponentially, manual execution is no longer viable. We need automation to ensure reliability and speed.

• Managing increased complexity
  Our system must be flexible and modular, allowing seamless configuration of new deployments and workflows.

• Ensuring system resilience
  We need self-healing mechanisms to automatically detect and resolve failures, reducing reliance on manual intervention.

• Streamlining customer rollouts
  Updates should be deployed seamlessly, ensuring our customers receive improvements in a controlled and efficient manner.

• Optimizing cloud infrastructure
  Expanding beyond a single provider ensures that we can optimize for cost, performance, and availability.

By focusing on these areas, we are not just solving problems—we are future-proofing our infrastructure for continued success.

How We’re Adapting

A truly scalable deployment pipeline should:

• Make Git the Source of Truth
  Deployments should be automated and reflect what’s in version control, eliminating manual intervention.

• Enable Blue/Green Deployments
  We should be able to deploy new versions of our software without taking down the current version, ensuring that unhealthy containers never replace healthy ones.

• Introduce Self-Healing Mechanisms
  If an application container or cloud resource fails, it should be identified and replaced automatically — without an engineer needing to step in.

• Automate Migrations
  Database migrations and configuration changes should be atomic, idempotent, and fully automated.

• Expand Multi-Cloud Capabilities
  Deploying across multiple cloud providers allows us to balance cost, performance, and availability.
  

The Technologies Enabling Our Growth

We are leveraging industry-leading tools to transform our deployment process:

• Kubernetes
  The foundation of our new infrastructure, providing automatic scaling, self-healing, and declarative configuration.

• Crossplane
  A Kubernetes-based infrastructure automation tool that allows us to manage multi-cloud environments seamlessly.

• ArgoCD
  A GitOps solution that ensures our deployments are fully automated and always in sync with version control.

• Observability Tooling
  There are many new tools available to improve the information delivered to developers and operators to help them perform their jobs. We will take a look at some of the foundational tools in the observability space, such as OpenTelemetry, Grafana, Prometheus, and Loki.

• Developer Portal
  To improve visibility into deployments and allow non-technical users to manage certain platform aspects. We are still looking at solutions that deliver this and will include our findings in a later part of this series.

• AI Agents
  The rapid pace of change in AI tooling is one of the biggest challenges and opportunities facing all branches of development, and infrastructure is no different. As we adopt AI tooling we will document the process here.

These aren’t just buzzwords—they’re the key components that allow us to scale without adding unnecessary complexity.

What's next?

In our next article, we’ll dive deep into Kubernetes and how it enables resilient, scalable scientific software deployments. We’ll walk through how the Kubernetes control loop can replace our existing deployment model and simplify the lives of our DevOps team.



After that, we’ll explore Crossplane, and ArgoCD, breaking down how these tools work together to create a seamless, automated deployment pipeline.

This series is about more than just technology—it’s about how we rethink the way scientific software is deployed and managed. If you’ve ever navigated the complexities of scaling a scientific SaaS platform, you’ll find insights here. Stick with us—we’re turning growth challenges into scalable solutions.