Latest updates for Web Experimentation

Experiment Summary

in development

Use the Summary page as the single source of truth for your experiment.

• Key components of an experiment will be surfaced in a summary view
• Save time during your QA, planning, or retro processes

Edge Delivery

released

The fastest way to deliver web experiences to users.

• CDN processing: Assign users to experiments and apply changes at the edge before your web page even reaches your visitors
• Boost page load: Less JavaScript delivered to your users means faster website performance
• Backwards compatibility: Implement Edge Delivery into existing Web projects without having to rebuild them

AI Variation Summary

released

Quickly understand your experience variations at a glance.

• Each variation's changes will be summarized into natural language for easy readability and better understanding
• Less manual work, no more clicking into individual variations to see detail

Experience Templates

released

Pre-built templates for web page components will allow marketers to spin up on-brand experiences for their personalization campaigns in a few easy steps.

• Use pre-built templates for components like count-down timers, banners, exit intent pop-ups, and more
• Select the template you want to use directly from the experience builder, where you're already working

Learn more

Event Properties

released

Expand your events with contextual information that support deeper metrics.

• Get deeper metrics: Create more meaningful metrics that can be filtered by additional custom data
• Simplify event capture: Capture similar but unique events, such as clicks on different elements of a header menu
• Easily categorize: Group multiple related events within a single event and filter them down on properties included with that event

Learn more

Google Analytics 4: Audience Targeting

Scale experimentation and personalization programs with a more powerful Audience Builder that allows you to deliver tailored experiences to your visitors more easily.

• Eliminate the need to rebuild segments, and run more targeted, impactful experiments.
• Retrieve Google Audiences from GA4: Pull audience data from GA4 and display a list of available Google Audiences in Optimizely’s Audience Builder so you can quickly target them
• Identify Google Audiences via a unique value/ID: Connect audiences sent to Optimizely with a snippet to validate that the visitor meets the Google Audience conditions for user bucketing

Here's how to enable the integration

Optimizely Reporting for Experimentation

Socialize the impact of your program and attract executive investment.

• Easily generate reports that are timely, valuable, and accessible to all levels of your organization.
• Out-of-the-box reports serve experimentation Program Managers, experimenting Marketers, and Data Analysts

• Metric Impact Report: See how each experiment contributed to increasing the target metric

• Program Throughput Report: Live view of % of visitors in at 0, 1, 2, 3, or N number of experiments

New Concluded Status

Conclude experiments, documenting end outcome and learnings for reporting.​

• Identify concluded experiments with a new "concluded" status and record your results (i.e. positive, negative, inconclusive)
• Make reporting easier and results documentation clearer

Optimizely Integration for Slack

Receive and manage real-time experiment notifications directly within Slack 

• Manage your projects and experiments seamlessly within Slack, a leading collaboration tool.
• Subscribe to projects and receive fast notifications, including statistical significance (Stat Sig) updates to stay informed about experiment outcomes and act promptly. 
• Access and engage with experiment results directly in Slack, reducing the need to log into multiple platforms. 
• Boost organizational efficiency, accelerate experimentation velocity, and drive greater awareness and satisfaction through a data-driven culture.  

CDP Audience Sync

Connect your 3rd party CDP for enhanced experiment, release targeting, and personalization campaigns.

• Leverage existing segments in your 3rd party CDP for experimentation and releases
• Bring Optimizely’s Real Time Segment (RTS) capabilities to your existing CDP​
• Single source of truth for audiences across your stack​

BigQuery Direct Share

Simplifying data analysis for BigQuery users.

• Real-time data available in your BigQuery
• Ability to combine datasets
• Query Optimizely Experimentation data however you like

Real-time Segments for Experimentation

Target experiments to user segments based on real-time behavior.

• Utilize pre-built segments in Optimizely Data Platform​
• Combine Real-time Segment targeting conditions with user attribute-based conditions within the audience builder for enhanced control​
• Target according to your software delivery strategy i.e beta, geo targeted or phased rollouts​

AI Variation Generator

In Web Experimentation's Visual Editor, you can select any text element within your page and generate AI-suggested copy based on your original text. You can then select one of those suggestions and automatically create a new variation for your experiment with one click.

• Preview suggestions in the Visual Editor before you confirm your selection
• Provide positive or negative feedback on individual suggestions
• Regenerate suggestions for a fresh list of ideas
• Convert a suggestion into a new variation with a single click

Single Sign-on, New Navigation, Admin Center

Optimizely One brings all of our products together to form a single, unified, cohesive solution. This gives practitioners the simplicity (and delight) of being able to toggle across different applications with ease, while giving Admins the power and flexibility to have all the controls they need directly at their fingertips.

• Consolidated Authentication: Streamlined and secure login with SSO and MFA controls
• Cross-Product Authorization: Seamless navigation across Optimizely apps
• Centralized Administration: Simplified user management with roles and permissions across Optimizely

These features are rolling out throughout 2024. Reach out to your Customer Success Manager to learn more.

Dynamic Selector Support

Web Experimentation's Visual Editor now supports single page applications via JavaScript frameworks, including React, AngularJS, NextJS, etc.

• Custom attribute-based targeting increases the reliability of dynamic selector targeting
• Accelerate your experiment configuration and reduce reliance on developers
• Launch experiments faster using our standard selector targeting without needing to manipulate your source code

Additional UX Delights

• Save Test URL(s) of a Page: Once a URL is added to the test URL(s) section, it will be saved upon navigating away to save QA time and avoid duplication of work
• ID column in Audiences/Extensions list page: Surface IDs for Audiences and Extensions on the list page for quicker references when using the REST API and parity with other components.
• Collapsible HTML section: Ability to collapse the HTML section in the Visual Editor for improved UX when editing/adding elements.
• ​Web UI cleanup: This includes removal of Optimizely Classic, deprecated audience conditions, deprecated Cross-Browser Testing, and fixing typos and redirect links.

Q1 2024 Deliveries

Dynamic Selector Support

In beta

Web Experimentation's Visual Editor now supports single page applications via JavaScript frameworks, including React, AngularJS, NextJS, etc.

• Custom attribute-based targeting increases the reliability of dynamic selector targeting
• Accelerate your experiment configuration and reduce reliance on developers
• Launch experiments faster using our standard selector targeting without needing to manipulate your source code

Sign up for the beta here!

Google Analytics 4: Audience Targeting

In beta

Scale experimentation and personalization programs with a more powerful Audience Builder that allows you to deliver tailored experiences to your visitors more easily.

• Eliminate the need to rebuild segments, and run more targeted, impactful experiments.
• Retrieve Google Audiences from GA4: Pull audience data from GA4 and display a list of available Google Audiences in Optimizely’s Audience Builder so you can quickly target them
• Identify Google Audiences via a unique value/ID: Connect audiences sent to Optimizely with a snippet to validate that the visitor meets the Google Audience conditions for user bucketing

Here's how to enable the integration

Automatic Sample Ratio Mismatch Detection

generally available

Set up a smoke alarm to safeguard your experiments! Identifying sample ratio mismatches is crucial, as about 10% of A/B test outcomes become unreliable due to significantly skewed visitor counts. Use Optimizely's automatic SRM detection to receive notifications when traffic distribution to a variation deviates greatly from intended proportions. Swiftly detect experiment issues and take action.

• Get an alert with information on why a mismatch is happening and how to fix it
• Increase the number of high-quality experiments you run
• Decrease the number of invalid business decisions you make

Read more detail in our support documentation

Knowledge Center

Generally Available

Knowledge Center offers you a seamless way to interact with Optimizely news and knowledge!

• Product roadmap updates
• New feature releases
• Join beta programs
• Submit product feedback

Take a peek at how it looks and feels

Statistical Significance Notifications

Generally available

With notifications for statistical significance, you can be confident in your experiment launches knowing that if a monitoring metric reaches statistical significance – commonly an unanticipated event, like a significant increase in bounce rate – you will be alerted in case intervention is required.

• Automated email alerts indicate when any metric in any variation in an experiment reaches statistical significance
• Reduce operational risk by being alerted when primary metrics reach statistical significance
• Increase experiment velocity by timely concluding experiments and launching next experiments

For further information, check out our support documentation

What's here now

Experiment Collaboration

Generally Available

Welcome to a better way to run experimentation, with one centralized place to: 

• Document the context of an experiment (hypothesis, variation designs, key learnings, etc), promoting collaboration and preserving discussion
• Track tasks in the design, build, and launch phases of an experiment to ensure experiments are launched on time and with qualtiy
• Customize workflows and codify your experimentation process to build teams that are self-sufficient…regardless of maturity
• Embed design files and directly annotate them in-app to simplify the feedback process and land on a winning design idea faster
• View and share your experimentation program however you want: a List, Calendar, Timeline, or Board view.

Learn more on the Experiment Collaboration Page, and try an interactive demo! To get access to these features, reach out to your Customer Success Manager or Account Manager.

Google Analytics 4: Report Generation

Generally Available

Connect your Web Experimentation project to the GA4 property you want to associate it with.

• Send specific (or all) variations within an experiment to GA4 as an audience
• A unique ID surfaces the data associated with each user in the variant

Here are the steps to set it up

Copy Generator

Generally Available

In Web Experimentation's visual editor, you can select any text element within your page and get AI-generated copy suggestions that are similar to your orginal text. Try it out!

• Preview suggestions in the visual editor before your confirm your selection
• Provide positive or negative feedback on individual suggestions
• Regenerate the suggestions if you want a fresh list of ideas​

Copy Generator is generally available as of November 2023!

Statistical Significance Notifications

Generally available

With notifications for statistical significance, you can be confident in your experiment launches knowing that if a monitoring metric reaches statistical significance – commonly an unanticipated event, like a significant increase in bounce rate – that you will be alerted in case intervention is required.

• Automated email alerts that indicate when any metric in any variation in an experiment reaches statistical significance
• Reduce operational risk by being alerted when primary metrics reach statistical significance
• Increase experiment velocity by timely concluding experiments and launching next experiments

For further information, check out our support documentation

What's coming soon

Automatic Sample Ratio Mismatch Detection

generally available - dec '23

Set up a smoke alarm to safeguard your experiments! Identifying Sample Ratio mismatches is crucial, as about 10% of A/B test outcomes are unreliable due to significantly skewed visitor counts. Use automatic SRM detection to receive notifications when traffic distribution to a variant deviates greatly from intended proportions. Swiftly detect experiment issues and take action.

• Get an alert with information on why a mismatch is happening and how to fix it
• Increase the number of high-quality experiments you run
• Decrease the number of invalid business decisions you make

Read more detail in our support documentation

Dynamic Selector Support

Alpha

Configure visual editor support for single page applications via JavaScript frameworks, including React and NextJS.​

• Reduce experiment configuration time
• Empower marketing teams to author experiments with less reliance on developers

More information to be announced soon!​

Winner Rollouts

Alpha

Roll out a winning variation without asking for help. 

• Easily roll out winning variations to all users at the push of a button
• Reduce the need for additional development resources

More information to be announced soon!

Experiment program reporting

in Development - Beta in Q1 '24

Give your experimentation program more visibility and attract more executive investment. Program Reporting makes it easier to aggregate and generate reports that are timely, valuable, and accessible to all levels of your organization.

• Out-of-the-box reporting for experimentation Program Managers

• Metric Impact Report: See how each experiment contributed to increasing the target metric

• Program Throughput Report: Live view of % of visitors in at 0, 1, 2, 3, or N number of experiments

Knowledge Center

open beta

Coming to Web Experimentation in late-November, Knowledge Center will offer you a seamless way to interact with Optimizely knowledge!

• Product roadmap updates
• New feature releases
• Join beta programs
• Submit product feedback

Take a peek at how it will look and feel.

That's all for now! Read our Release Notes for information on all releases.

Google Analytics 4: Audience Targeting & Report Generation

This out-of-the-box integration requires just 1 click to connect the two systems. Web Experimentation users can now integrate with Google Analytics 4 to send their experiment data to Google, send variation viewers to GA4 as audiences, and more.

• Easy Authentication: Connect Web Experimentation and Google Analytics 4 instances with a single click integration that allows you to send visitor events directly to GA4.
• Report Generation: Connect your Optimizely Web Experimentation project to the GA4 property you want to associate it with. Report Generation is Generally Available as of October 2023.
• Audience Targeting: Coming later this year, you can create audience segments (based on specific user behaviors and demographics, such as age, gender, location, and more) from within GA4, and send it directly to Web Experimentation.

For information on how to set it up, refer to our developer documentation.

Introducing: Experiment Collaboration

Scaling an experimentation program requires coordination and planning that can burden existing resources and require additional headcount. Though existing planning tools can help, they aren't purpose-built for experimentation and lack deep integration with Optimizely. With Experiment Collaboration, your team can: 

• Document all parts of an experiment (hypothesis, variations assets, and key learnings), promoting collaboration and preserving discussion  
• Track tasks in the design, build, and launch phases of an  experiment to ensure experiments are launched on time 
• Collaborate on test planning templates to prompt meaningful discussion upfront, and keep that discussion in one place
• Create workflows and codify the experimentation process to build teams that are self-sufficient…regardless of maturity
• Embed design files and directly annotate screenshots to make it easier to collect feedback and land on a winning idea
• Better manage your experimentation program by using a variety of planning visualizations including calendars, timelines, and board views 

Experiment Collaboration is Generally Available as of September 2023. If you're interested in seeing and learning more, reach out to your account manager or drop us a Contact Us.

Read more details in this announcement written by the Principal Product Manager of our Experiment solution, Brad Yee.

Advanced Audience Targeting

Experimenters depend on audience integrations with third-party customer data platforms to extend experimentation and personalization targeting capabilities.

• Use ODP real-time segments out-of-the-box in your experiments, personalization campaigns, and targeted deliveries
• Use existing segments from your CDP (mParticle, etc)
• Build integrations with custom data sources through Optimizely Connect Platform
• Anonymous Targeting: Target users before they log in, and personalize their experience
• (Coming Soon) Upload lists of users to target

Now you can collect valuable visitor behavioral data, create hyper-specific customer segments, and unlock new use cases for advanced audience targeting, for example: experimenting on visitor cohorts such as a marketing list subscriber, past purchaser, or highly engaged shopper.

Advanced Audience Targeting is in Private Beta, you can request access here.

For more information, ask your Customer Success Manager or check out our developer documentation 

Real-Time Segments

Briefly noted above, for those of you using ODP (Optimizely Data Platform), Real-Time Segments will allow you to build custom audiences for real-time personalization and target those audiences with a personalized experience through our Experimentation products, using the freshest data available. For example, you could know about someone abandoning their cart in <90 seconds, and then start personalizing their experience.

• Laser-Focused Targeting: Define custom audiences based on demographics, browsing behavior, and past purchases for deeply personal connections and increased brand loyalty
• Agile Adaptation: Respond to changing trends and opportunities by tailoring campaigns on the fly, maximizing marketing impact and staying ahead of the curve
• Enhanced Customer Experiences: Deliver extraordinary experiences with real-time personalization, offering relevant content and offers that leave a lasting impression
• Increased ROI: Drive higher engagement, conversions, and bottom-line growth with Real-Time Segments' personalized messages, making every marketing effort a strategic investment

If you don't have ODP, you can request access to ODP Lite's beta here and be notified when you're able to try it out!

Statistical Significance Notifications

With notifications for statistical significance, you can be confident in your experiment launches knowing that if a monitoring metric reaches statistical significance – oftentimes an unanticipated event, like a significant increase in bounce rate – that you will be alerted in case intervention is required.

• Automated email alerts that indicate when any metric in any variation in an experiment reaches statistical significance
• Reduce operational risk by being alerted when primary metrics reach statistical significance
• Increase experiment velocity by timely concluding experiments and launching next experiments

Statistical Significance Notifications are Generally Available as of September 2023, you can sign up for access here and if you want further information, check out our developer documentation.

Shadow DOM support

Companies that architect their websites using shadow DOMs usually do so to compartmentalize their work and avoid conflicting changes on the same webpage. 

• Allows hidden DOM trees to be attached to elements in the regular DOM tree
• Traverse your shadow tree in the Visual Editor and select elements within it
• Create boundless experiments throughout your site

Click the checkbox in your Settings (pictured below) to enable this feature!

Automatic Sample Ratio Mismatch Detection

Roughly 10% of A/B tests run are invalidated because visitor counts in one of the variants has diverged significantly from the intended proportions, causing a Sample Ratio Mismatch. 

• Understand the pervasiveness across our clients and why they occur
• Decrease the number of invalid business decisions customers make
• Get an SRM alert, information about why an SRM is happening and how to fix it
• Increase the number of high-quality experiments

Automatic Sample Ratio Mismatch Detection is Generally Available as of October 2023. Read more detail in our support documentation.

Flexible Experiment List Columns

With flexible list columns, customize the experiment list page by adding new columns to improve organization. For example, choose to view experiments by:  

• Days Running
• Primary Metrics used  
• The Experiment Creator 

Each customized view has a unique URL so that they can be bookmarked in the browser and shared. This feature is Genearlly Available as of July 2023, you can show/hide columns by clicking on the column button in the top-right corner (pictured below). 

Mutually Exclusive Experiments (for Edge)

When running a Performance Edge experiment, you can now provide “exclusion groups” to make those experiments mutually exclusive.

• Get non-biased results more often
• Increase the number of concurrent experiments that you’re able to run
• Isolate related experiments that are targeting related audiences
• Use when you don't want a visitor to see multiple experiments at the same time.

Learn more in our developer documentation

UX Delights

• In-product descriptions for URL match types - Differentiate the URL match types to guide users in selecting the appropriate one. 
• Show description in an archived experiment - Display the description of an experiment even when it’s been archived.
• Trigger Simpson’s Paradox Warning message - Only display Simpson’s Paradox warning message when variation traffic distribution, not total traffic allocation, is modified in a running experiment to avoid confusion.
• New advanced project-level SPA settings - Provide new advanced project-level SPA settings for automatic page deactivation and undoing of changes affecting all newly created Pages, therefore automating the need for manual configuration per Page.
• SPA Page Trigger Alert - Warn users instantly when they select a page trigger that requires dynamic website support and the project setting is disabled, therefore reducing the possibility of experiment bugs.
• Visual Editor enhancements - Modify message displayed for more intuitiveness when the Page does not load in the Visual Editor and make ‘delete’ button available so the user is still able to delete the change.

For more information on all recent releases, read our Release Notes