We rely on many tools at Magic Patterns to ship new features & run efficiently. Here are some of our favorites that we can't live without.
Whalesync
Whalesync helps us turn a simple spreadsheet into the Magic Patterns control center. We are constantly running experiments, feature flags, and routing traffic.
Without Whalesync, we would have to build internal tools to help manage these things.
We're excited to use some of Whalesync's other features in the future to run A/B tests on our landing page, control content through our app, and replace more of our internal tools.
The most common thing we modify on a regular basis is which model powers Magic Patterns. We are continuously running tests to see which model is performing the best!
What we love about Whalesync:
- It completely replaces any internal tools we would have to build
- It's a delightful product that's easy to set up and use
- Easy to get up and running, but infinitely flexible
Langfuse
Langfuse has been our go-to tool for logging and monitoring our AI outputs.
LLMs tend to be a bit volatile and unpredictable sometimes, so Langfuse has been critical for helping us debug and improve our prompts and processing steps.
The coolest part? They are fully open-source and have an amazing community. If you're building a LLM product, this is a must-have.
What we love about Langfuse:
- Has helped us build a better and more stable AI product
- Makes debugging outputs from models 100x easier
- Intuitive UI/UX
Relace
Relace has been a critical tool for us to create custom, design-focused AI models. Current popular foundation models (like GPT and Claude) are great, but we want to provide a more tailored experience for our customers at Magic Patterns. For example, LLM speed is a key feature we optimize for since design is all about iterating!
Relace has allowed us to fine tune on our own data and continuously create better and better models.
Not every model works out great, but it's been a lot of fun to experiment and see what works well.
What we love about Relace:
- Takes all the headache out of fine tuning
- Helps reduce costs by using open-source models
- Automatic collection of labeled data
Wrapping Up
These tools have become essential parts of our infrastructure at Magic Patterns, each solving specific problems in elegant ways.
If you're building a LLM product, we would highly recommend checking out these tools.