A Custom Python App to Speed Up Salesforce Marketing Cloud Lookups

If you’ve spent time working with Salesforce Marketing Cloud (SFMC), you’re probably familiar with Contact Builder and the process of mapping Data Extensions (DEs) to perform lookups. While Contact Builder is a powerful tool, it can also be slow and cumbersome, especially when you’re trying to do something as simple as checking a single value across multiple DEs. In this post, I’ll share a custom Flask app I built that offers a faster, more flexible alternative to Contact Builder for multi-DE lookups.

The Problem with using Contact Builder for Lookups

Using Contact Builder for lookups can be a frustrating experience, particularly when:

• Performance is slow: Contact Builder isn’t exactly known for its speed. Loading the UI, mapping Data Extensions, and then running lookups can take far more time than you’d like.
• Mapping limitations: To use Contact Builder, you have to map each Data Extension you want to work with. This might be fine for a few key DEs, but when you’re dealing with temporary or test DEs, this step becomes impractical—especially in a production environment where data may be sensitive or frequently changing.
• Overkill for simple tasks: If you’re just trying to look up a single value across multiple Data Extensions, Contact Builder can feel like using a sledgehammer to crack a nut. It’s designed for complex relationships and segmentation, but not for quick, one-off lookups.

The Simple Alternative: A Custom Lookup App

To solve the issues with Contact Builder, I created a custom Flask app that allows you to perform fast lookups across multiple Data Extensions with just one input value. Here’s how it improves on Contact Builder:

• Faster lookups, fewer steps: Instead of waiting for Contact Builder to load, this app allows you to submit a lookup value and get your results in seconds.
• No mapping required: The app reads from a simple lookups.json file that lists the Data Extensions and the fields you want to search. You don’t have to map these DEs in Contact Builder, which makes it perfect for production environments or when working with temporary test data.
• Works with any DE: Whether you’re working with live production data or test Data Extensions, the app can retrieve the information you need without the hassle of Contact Builder.

How the App Works

The app is built using Flask and connects to SFMC using the SOAP API. Here’s a quick look at how it works:

1. Configuration: You define the Data Extensions and the lookup fields in a simple JSON file (lookups.json).
2. User Input: The app provides a simple web form where you enter a single lookup value (like a SubscriberKey or an OrderNumber).
3. Results: The app retrieves the data from each Data Extension listed in lookups.json and displays it in a clean table format.

Example Use Case

Imagine you’re trying to find all instances of a subscriber using their SubscriberKey. Instead of having to map every relevant Data Extension in Contact Builder, you can define these DEs in the app’s lookups.json file, enter the SubscriberKey in the form, and get instant results—without any additional configuration.

Setting It Up

Setting up the app is straightforward. You can clone the repo from GitHub and follow these steps:

1.Clone the Repository: Start by cloning the repository:

git clone https://github.com/lucasCostaYVR/sfmc-helper.git
cd sfmc-helper

2.Set Up a Virtual Environment: It’s best to isolate your dependencies with a virtual environment:

python3 -m venv venv
source venv/bin/activate  # On macOS/Linux
venv\Scripts\activate     # On Windows

3. Install Dependencies: Once your virtual environment is active, install the required Python packages:

pip install -r requirements.txt

4. Create a .env File: You’ll need to create a .env file with your Salesforce Marketing Cloud credentials and base URLs. Here’s an example format:

CLIENT_ID=your-client-id
CLIENT_SECRET=your-client-secret
AUTH_BASE_URL=https://your-auth-url.auth.marketingcloudapis.com/
REST_BASE_URL=https://your-rest-url.rest.marketingcloudapis.com/
SOAP_BASE_URL=https://your-soap-url.soap.marketingcloudapis.com/
FLASK_ENV=development

5. Edit lookups.json: Customize the lookups.json file to include the Data Extensions you want to search. It should look something like this:

[
    {
        "ExternalKey": "SubscriberKeyLookup",
        "LookupField": "SubscriberKey"
    },
    {
        "ExternalKey": "OrderLookup",
        "LookupField": "OrderNumber"
    }
]

6. Run the App: Now you can run the Flask app:

python app.py

Why This App Saves Time

• Faster lookups: With this app, you don’t have to wait for Contact Builder to process your query. Just input the value, and the results come up almost instantly.
• No need to map DEs: One of the biggest time-savers is the ability to work with Data Extensions directly, without having to map them in Contact Builder. This makes it easier to work in production environments, where you may not want to set up mappings for every DE.
• Perfect for production and test environments: Whether you’re troubleshooting live production data or experimenting with test Data Extensions, the app gives you flexibility that Contact Builder simply doesn’t.

Limitations and Next Steps

This app was built with the goal of simplifying lookups in Data Extensions, making it a handy tool for daily activities, data investigations, and QA. However, there are a few limitations to keep in mind:

• Performance on large datasets: As the number of Data Extensions grows, the lookup process may become slower. This is because each lookup is performed sequentially, which can lead to delays when working with a large number of DEs.
• Single-value lookups only: Currently, the app focuses on performing lookups using a single value. More complex queries and filtering options aren’t supported yet.

Next Steps

To address the performance issue, I plan to make the lookup process asynchronous, allowing all lookups to be executed in parallel. This should significantly speed up the process, especially when dealing with multiple DEs.

Stay tuned for updates! I’ll be adding more features to the SFMC Helper app to enhance its usability and flexibility.

Conclusion

If you’ve been frustrated with the slowness or limitations of Contact Builder, this Flask app could be a game-changer. It’s fast, flexible, and allows you to perform lookups across multiple Data Extensions in your SFMC environment without the need for complex mappings. Whether you’re working with production data or conducting quick tests, this app can streamline your workflow.

While there are still some limitations, like performance with larger DEs, future updates will bring asynchronous lookups to make the app even faster. Give it a try, and feel free to share your feedback—your input will help shape the next round of features!