Use Databricks as a source — pulling events for reporting

Create Kameleoon goals that pull conversion data from Databricks on a scheduled basis to track experiment results using your warehouse data.

Once you’ve activated Databricks for a specific project, you can use it to create goals in Kameleoon. These goals utilize conversions data directly from your Databricks database.

To create a goal using Databricks, follow the Data warehouse tracking flow described in Create a goal, and choose Databricks from the Datawarehouse integration dropdown. Step 1 then asks for these Databricks-specific fields:

• Frequency: Select how often the task should run.
• Catalog: Enter the name of the Databricks catalog that contains the schemas you wish Kameleoon to read from.
• Schema: Enter the schema that contains your conversion data.
• Query: Define the SQL query to retrieve the necessary data from Databricks.

Once Step 1 is complete, configure Step 2, Advanced settings, if needed, then save your goal.

Data warehouse retention period: For Kameleoon to poll an event, it must remain accessible by your input query for at least 72 hours after the event occurred.

Query format

The query must adhere to a specific format:

SELECT visitor_id, conversion_timestamp FROM your_events_table

Where visitor_id is the column representing your visitors’ unique ID, and conversion_timestamp is a column representing the exact time at which the conversion took place. In Databricks, the conversion_timestamp column must be a Timestamp type column.

If you want to associate a revenue to each conversion, the query should adhere to an alternate format:

SELECT visitor_id, conversion_timestamp, revenue FROM your_events_table

Where revenue is a column containing the revenue for each conversion.

For more complex queries you can adhere to this format by formulating a sub-query as such:

SELECT visitor_id, conversion_timestamp, revenue FROM ( {your_original_query} ) AS subquery

Your query runs every hour in your Databricks warehouse, with an added WITH clause that filters by timestamps. Keep in mind that although Kameleoon collects conversions hourly, it only merges them into your experiment results once per day.

Run your query before ingestion

Before saving your ingestion task, you can test your query directly in Kameleoon. Testing allows you to:

• Verify the connection in real time.
• Confirm your credentials and access rights are correct, which helps detect issues immediately, without having to wait for the first data import.
• Validate your data’s structure and accessibility.

You can run a simple query to ensure Kameleoon can access the right databases and tables. You can also download a small sample of the dataset to confirm that the schema, permissions, and contents match your expectations.

Using your Databricks goal

Now that you’ve created a goal, you can incorporate it into your Kameleoon campaigns. When setting up an experiment or personalization, you can select the goal, allowing you to track and analyze specific conversions in your Databricks database. Kameleoon merges these conversions with your experiment results once per day, so all conversions polled on a given day are available the following morning.

To learn how to set up a goal in an experiment, see Set up a goal in a campaign.