In-depth: PostHog vs GrowthBook

Oct 10, 2023

PostHog and GrowthBook both provide open source, self-serve feature flags and experimentation, but they're different in two important ways:

• GrowthBook is a warehouse-native feature flag and experiments platform. It focuses on integrating with the product and data tools you already use.

• PostHog is an all-in-one product and data platform. Beyond feature flags and experiments, it includes product analytics, session replay, surveys, CDP, and more.

In this post, we'll cover these differences in more detail, and answer frequently asked questions about both tools.

How is PostHog different?

1. PostHog is an all-in-one platform

PostHog brings together all the tools engineers need for testing, releasing, and measuring the success of new features. Feature flags and A/B testing are only part of a suite of tools PostHog offers.

PostHog combines usage, performance, and behavioral data with flags and experiments. PostHog’s data warehouse also enables you to pull in data from external sources.

Having all these product and data tools together enables you to do better analysis of shipped features and make better decisions about what you are building next.

2. PostHog is built for startups and engineers

PostHog is built for high-growth startups. This means it is simple for founders and engineers to implement themselves. There are many SDKs, tutorials, and docs to help you get started quickly with any type of app.

As startups scale, PostHog also provides the more advanced tools they need to succeed. These include advanced product analytics, SQL querying, CDPs, and data warehousing.

GrowthBook, on the other hand, focuses only on later-stage, larger companies than PostHog. Many of their features, like their analytical A/B testing suite, are great for data teams which also come at a later stage.

3. PostHog is easier to setup

GrowthBook requires more setup than PostHog as it relies heavily on external tools and writing SQL:

1. To get data into GrowthBook, you must integrate a data source.
2. To track data related to flags and experiments, you connect to an analytics tool.
3. To configure data sources and track events, you write SQL configurations in GrowthBook.
4. To set up goals and analysis, you write more SQL.

PostHog needs none of this. You use the same app and SDKs for managing and evaluating feature flags as tracking usage. There is no extra configuration needed. Creating flags, experiments, and insights doesn’t require writing SQL. This makes PostHog faster to get started and easier to use once set up.

Comparing PostHog and GrowthBook

Platform

Both PostHog and GrowthBook have the infrastructure to use flags and experiments effectively with your current app. PostHog has a wider range of product and data tools built in, while GrowthBook relies on third-party integrations

Feature flags

Both PostHog and GrowthBook offer all the functionality you expect from feature flags.

• Targeting: GrowthBook's custom targeting using attributes must be set every session and defined in-app before use. It's also only available on the paid Pro plan. PostHog automatically sets its equivalent (properties) on users. You don’t need to pre-define them and unlimited custom values are free.

• Bootstrapping: PostHog’s JavaScript web SDK enables you to pass flags directly from the backend before the app loads. This ensures they're available immediately and prevents flickering. GrowthBook instead recommends moving the A/B test or flag logic earlier in the page load (server-side) to prevent this.

Experimentation

Experimentation is where PostHog and GrowthBook’s functionality diverges. Both enable you to create A/B/n tests with custom goals and calculate statistical significance, but:

• GrowthBook’s A/B testing provides more potential statistical depth, such as Bayesian and Frequentist engines, value capping, regression adjustment, and testing corrections. These do require statistics expertise to use reliably, though.

• PostHog provides a simpler interface and no need to write SQL or connect to external services – a better experience for users without a background in data science.

• Recommended run time: PostHog automatically calculates a recommended run time based on past data and minimally acceptable improvements. This helps you avoid the peeking problem and end your experiment at the right time.

• Visual editor: GrowthBook’s visual editor is similar to PostHog’s toolbar. It enables you to select an element to modify using an A/B test. The editor is limited to server-side rendered apps.

• Frequentist engine: GrowthBook has an additional statistical analysis option. It enables you to compare the control with the experiment using p-values and confidence intervals.

Reporting and analytics

GrowthBook offers no reporting or analytics tools other than the ability to query experiment reports with SQL. As a result, GrowthBook forces users to rely on external event tracking and visualization tools.

PostHog provides all the visualizations and product tools for evaluating the success of your features and app. These include trends, funnels, retention, SQL querying, and session replays integrated with your flags and A/B testing data. This enables you to do deeper analysis of their impact and combine them with other product and usage data.

Pricing

PostHog’s feature flag pricing is pay-per-request (and A/B tests use feature flags). There is a generous free tier of 1M requests per month with all features, add-ons, and integrations available.

Like PostHog, GrowthBook is free to self-host. GrowthBook Cloud is a seat-based model. It is free for up to 3 users, then $20/user/month. They claim this provides "unlimited traffic," but in reality, the limit is 10M requests per month, after which you need an enterprise plan.

Features, like flag scheduling, permissions, custom fields, and the visual editor are only available on the GrowthBook Pro paid plan.

Example scenarios

To give you an idea of what pricing looks like in reality, here are some example situations and their estimated costs for both PostHog and GrowthBook.

Note: GrowthBook does not display their Enterprise pricing needed to go beyond 10M requests per month.

Notes:

• Using backend local evaluation in PostHog lowers the amount of flag usage depending on the polling duration and active number of servers. If you use locally evaluated flags with one server polling every 30 seconds, this amount is under 1M requests (free).
• PostHog has volume discounts on flags over 2 million requests per month.

Integrations

GrowthBook has integrations with data warehouses and analytics tools they rely on, but little beyond that. PostHog has a wider range of integrations.

PostHog’s event-based structure enables you to import data from anywhere for use with flags and experiments. The free API enables you to connect, edit, and capture from anywhere too. For example, you can import data from warehouses, no-code site builders, Segment, and more.

Security and compliance

Both PostHog and GrowthBook enable companies to remain secure and compliant with privacy regulations. Companies can customize the levels of user privacy related to these platforms to their needs.

Frequently asked questions

Who is PostHog useful for?

PostHog is built for startups and their engineers. It provides all the tools startups need to build successful products. The people who find PostHog most useful are founders, product engineers, and growth engineers.

Companies that use PostHog feature flags and experiments include Y Combinator, Vendasta, and AssemblyAI.

Who is GrowthBook useful for?

Larger organizations that want their feature flag and experimentation tools use GrowthBook. They often already set up data warehouses and analytics tools. GrowthBook is an added piece to their toolkit. In these organizations, it is used by a combination of engineering, data, and product teams.

Companies that use GrowthBook include Patreon, Deezer, and Pepsi.

How much does PostHog cost?

Feature flags and experiments are free for up to 1M requests per month. Beyond that, it costs $0.0001/request (or $1 per 10,000 requests). There are discounts for high-volume users, non-profits, and startups.

Other products, like product analytics and session replay, have separate but similarly structured pricing.

How much does GrowthBook cost?

GrowthBook, like PostHog, is free to self-host. This doesn’t count infrastructure, implementation, or maintenance costs.

GrowthBook Cloud is free for up to 3 users and 10M requests per month. Its "Pro" plan is $20/seat/month and provides access to premium features. They also have an Enterprise plan that adds more security and compliance tools as well as a higher request limit.

Do GrowthBook or PostHog offer free trials?

GrowthBook is free to self-host and has a free cloud tier with unlimited flags and experiments for up to 3 users.

PostHog lets you use all its features for free. Once you upgrade to paid, you get 1M flag requests, 1M events, and 5,000 recordings for free each month. If your usage stays below this, PostHog remains free.

How long does it take to implement PostHog?

Feature flags and experiments are simple. They are a few lines of code in any of your favorite languages. They can even be set up on no-code site builders like Framer or Webflow.

The process requires signing up for PostHog, installing the snippet or SDK in your app, creating the flag in PostHog, and implementing the flag evaluation and relevant logic in your app. Many of the SDKs handle important aspects like local evaluation and event capture for you.

You can reuse the PostHog implementation, like user identification, across products. Because PostHog is an all-in-one platform, analytics capture for targeting and A/B testing results doesn’t need set up or connection either.

How long does it take to implement GrowthBook?

The basic GrowthBook implementation seems relatively similar to PostHog, but the need to define data twice as well as integrate external tools makes it longer.

To implement GrowthBook from scratch:

• Sign up or self-host.
• Create an SDK endpoint, install the SDK in your app, and initialize the SDK.
• Set up target attributes in-app and define them in GrowthBook.
• Set up an analytics tool to track events and integrate it with GrowthBook.
• Connect your data source (warehouse) and write SQL to configure it.
• Create a feature flag or experiment. Set up tracking metric(s) using SQL. Implement related logic in-app.

Although GrowthBook provides templates for integrating external tools, this whole process takes longer than PostHog.