This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Read More
Lead scoring is one of those things that marketing ops teams set up, constantly maintain, and still lose sleep over.
The concept is simple. You assign points to leads based on who they are and what they do.
Open an email? +5. Download a whitepaper? +10. VP title at an enterprise company? +15. Hit a threshold, and the lead gets passed to sales as an MQL.
It sounds clean on paper. But in practice, it’s a different story.
Today, we’re going to talk about why rules-based lead scoring struggles to capture real buying behavior, and how you can build an AI-powered scoring model (fine-tuned on your own data) that actually reflects what a good lead looks like for your business.
We’ll walk through the full technical setup, including code snippets you can adapt for your own implementation.
Let’s get into it.
For those who need a refresher, lead scoring is a method for modeling lead engagement and fit. It’s how marketing tries to answer the question: “Is this person ready for a sales conversation?”
Most marketing automation platforms, like Marketo, support two dimensions of scoring:
Demographic scoring looks at who the lead is. Job title, company size, industry, and geography are the usual suspects. A Director at a 500-person SaaS company might score higher than an intern at a 10-person nonprofit, assuming you’re selling enterprise software.
Behavioral scoring looks at what the lead does. Email opens, link clicks, webinar attendance, page visits, form fills. Each action adds points, and the accumulation of those points is supposed to tell you how engaged someone is.
When a lead crosses a predefined threshold (say, 100 points), they become an MQL and get routed to sales.
The idea is solid, but the execution is where things break down.
Here’s the core problem: rules-based scoring treats every action in isolation. It can’t model the relationships between behaviors. And it definitely can’t account for the complex, nonlinear ways that real people actually buy.
Consider these scenarios:
A lead downloads three whitepapers in one day. Your scoring model adds +30 points and flags them as highly engaged. But maybe they’re a student doing research for a thesis. They’ll never buy anything.
Meanwhile, a VP visits your pricing page once, looks at a case study, and then goes quiet for two weeks. Your model barely notices them. But that VP just got budget approval and is about to reach out to your sales team.
Rules-based scoring can’t distinguish between these two leads because it doesn’t understand context. It just adds up numbers.
This leads to a few painful outcomes:
Low conversion rates on MQLs. Sales gets a pile of “qualified” leads that don’t actually convert. Over time, they stop trusting the scores entirely.
A strained marketing-sales relationship. Sales says the leads are bad. Marketing says sales isn’t following up properly. Nobody’s happy, and the finger-pointing goes nowhere.
Wasted time and budget. Your team spends hours tweaking point values (is a webinar worth 15 or 20 points?), but the fundamental limitation remains: A rules engine can’t learn patterns the way a model trained on real outcomes can.
If you’ve been in MOPs for a while, this probably sounds painfully familiar.
So what does AI-based lead scoring actually look like?
Instead of manually defining rules, you train a model on your own historical data. Specifically, you look at the leads you’ve already passed to sales over the last 3 to 6 months, and you identify which ones turned out to be great and which ones turned out to be duds.
Here’s the general process:
The result is a scoring model that doesn’t just add up points. It understands that a pricing page visit from a Director at a mid-market SaaS company, combined with a webinar attendance three days earlier, is a fundamentally different signal than the same visit from someone with no other engagement. It picks up on the patterns that rules can’t.
Let’s get technical. Here’s how to put this together.
You’ll need two things for each lead in your training set: their demographic profile and their activity history. Marketo’s REST API gives you both.
Pulling lead demographics:
import requests
MARKETO_BASE_URL = "https://your-instance.mktorest.com"
ACCESS_TOKEN = "your_access_token"
def get_lead_by_id(lead_id):
url = f"{MARKETO_BASE_URL}/rest/v1/leads.json"
params = {
"access_token": ACCESS_TOKEN,
"filterType": "id",
"filterValues": lead_id,
"fields": "email,firstName,lastName,title,company,numberOfEmployees,industry,country,leadScore"
}
response = requests.get(url, params=params)
return response.json()["result"][0]
Pulling the activity log:
For behavioral activity, you’d call the Get Lead Activities endpoint with the relevant activity type IDs (email opens, web page visits, form fills, and so on):
def get_lead_activities(lead_id, next_page_token):
url = f"{MARKETO_BASE_URL}/rest/v1/activities.json"
params = {
"access_token": ACCESS_TOKEN,
"leadId": lead_id,
"nextPageToken": next_page_token,
"activityTypeIds": "1,2,3,6,7,8,10,11,12,13,46"
# Common types: Visit Page, Click Link, Fill Out Form,
# Send Email, Email Delivered, Email Opened, Click Email,
# Add to List, Change Score, Change Data Value,
# Interesting Moment
}
response = requests.get(url, params=params)
return response.json()
To get the nextPageToken, you first need to call the paging token endpoint:
def get_paging_token(since_date):
url = f"{MARKETO_BASE_URL}/rest/v1/activities/pagingtoken.json"
params = {
"access_token": ACCESS_TOKEN,
"sinceDatetime": since_date # e.g., "2025-10-01T00:00:00Z"
}
response = requests.get(url, params=params)
return response.json()["nextPageToken"]
Collect this data for all the leads in your training set. You’ll want a solid sample: aim for at least 100-200 leads if possible, split between your best and worst performers.
This part requires human judgment. Go through your leads and assign each one a grade from 1 to 10 based on actual outcomes.
Here’s a rough grading guide you can adapt:
OpenAI’s fine-tuning API expects your data in a specific JSONL (JSON Lines) format. Each line is a conversation with a system prompt, a user message (containing the lead data), and an assistant response (the grade and reasoning).
import json
def format_training_example(lead_demographics, activity_log, grade, reason):
return {
"messages": [
{
"role": "system",
"content": (
"You are a lead scoring assistant for a B2B SaaS company. "
"Given a lead's demographic information and their Marketo "
"activity history, you score the lead from 1 to 10 and "
"provide a brief reason for your score. A score of 10 means "
"the lead is extremely likely to close. A score of 1 means "
"the lead is extremely unlikely to close."
)
},
{
"role": "user",
"content": (
f"Score this lead:\n\n"
f"Demographics:\n{json.dumps(lead_demographics, indent=2)}\n\n"
f"Activity Log:\n{json.dumps(activity_log, indent=2)}"
)
},
{
"role": "assistant",
"content": (
f'{{"score": {grade}, "reason": "{reason}"}}'
)
}
]
}
# Write all training examples to a JSONL file
with open("training_data.jsonl", "w") as f:
for example in training_examples:
f.write(json.dumps(example) + "\n")
With your training file ready, you can kick off the fine-tuning job:
from openai import OpenAI
client = OpenAI(api_key="your_api_key")
# Upload training file
training_file = client.files.create(
file=open("training_data.jsonl", "rb"),
purpose="fine-tune"
)
# Create fine-tuning job
fine_tune_job = client.fine_tuning.jobs.create(
training_file=training_file.id,
model="gpt-4o-mini-2024-07-18" # Cost-effective base model for fine-tuning
)
print(f"Fine-tuning job created: {fine_tune_job.id}")
The fine-tuning process typically takes anywhere from 15 minutes to a couple of hours, depending on the size of your dataset. Once it completes, you’ll get a custom model ID that you can use in API calls just like any other OpenAI model.
You can check the status of your job:
job_status = client.fine_tuning.jobs.retrieve(fine_tune_job.id)
print(f"Status: {job_status.status}")
print(f"Fine-tuned model: {job_status.fine_tuned_model}")
Now for the fun part: making this work in real time.
The architecture looks like this:
This is important:
Do not use a “Response to Webhook” node to send data back to Marketo.
Webhook responses have strict time limits (usually a few seconds). Your workflow needs time to make multiple API calls to Marketo, send data to OpenAI, and wait for a response from a reasoning model. That chain of operations can easily take 10-30 seconds, which will cause a webhook timeout.
Instead, treat the webhook as a one-way trigger. Let it fire and forget. Then, at the end of your iPaaS workflow, call the Marketo REST API Leads endpoint directly to write the score and reason back to the lead record.
Here’s what the write-back looks like:
def update_lead_score_in_marketo(lead_id, ai_score, ai_reason):
url = f"{MARKETO_BASE_URL}/rest/v1/leads.json"
headers = {
"Authorization": f"Bearer {ACCESS_TOKEN}",
"Content-Type": "application/json"
}
payload = {
"action": "updateOnly",
"lookupField": "id",
"input": [
{
"id": lead_id,
"AI_Lead_Score__c": ai_score,
"AI_Score_Reason__c": ai_reason
}
]
}
response = requests.post(url, headers=headers, json=payload)
return response.json()
If you’re building this in n8n, your workflow will look something like this:
Node 1: Webhook Trigger Receives the lead ID from Marketo when a score-relevant change occurs.
Node 2: Get Marketo Access Token
POST https://your-instance.mktorest.com/identity/oauth/token
?grant_type=client_credentials
&client_id=YOUR_CLIENT_ID
&client_secret=YOUR_CLIENT_SECRET
Node 3: Get Lead Demographics
GET https://your-instance.mktorest.com/rest/v1/leads.json
?filterType=id
&filterValues={{leadId}}
&fields=email,title,company,numberOfEmployees,industry,country
Node 4: Get Paging Token
GET https://your-instance.mktorest.com/rest/v1/activities/pagingtoken.json
?sinceDatetime=2025-01-01T00:00:00Z
Node 5: Get Lead Activities
GET https://your-instance.mktorest.com/rest/v1/activities.json
?leadId={{leadId}}
&nextPageToken={{pagingToken}}
&activityTypeIds=1,2,3,6,7,8,10,11,12,13,46
Node 6: Call Fine-Tuned OpenAI Model
import json
# This runs in n8n's Code node
demographics = $input.all()[0].json.demographics
activities = $input.all()[0].json.activities
response = await fetch("https://api.openai.com/v1/chat/completions", {
method: "POST",
headers: {
"Authorization": "Bearer YOUR_OPENAI_KEY",
"Content-Type": "application/json"
},
body: JSON.stringify({
model: "ft:gpt-4o-mini-2024-07-18:your-org::your-fine-tune-id",
messages: [
{
role: "system",
content: "You are a lead scoring assistant for a B2B SaaS company. Given a lead's demographic information and their Marketo activity history, score the lead from 1 to 10 and provide a brief reason. Respond only in JSON format: {\"score\": <number>, \"reason\": \"<text>\"}"
},
{
role: "user",
content: `Score this lead:\n\nDemographics:\n${JSON.stringify(demographics)}\n\nActivity Log:\n${JSON.stringify(activities)}`
}
],
temperature: 0.2
})
});
const result = await response.json();
const scoring = JSON.parse(result.choices[0].message.content);
return { score: scoring.score, reason: scoring.reason };
Node 7: Update Lead in Marketo (via API, NOT webhook response)
POST https://your-instance.mktorest.com/rest/v1/leads.json
Body:
{
"action": "updateOnly",
"lookupField": "id",
"input": [{
"id": "{{leadId}}",
"AI_Lead_Score__c": "{{score}}",
"AI_Score_Reason__c": "{{reason}}"
}]
}
This is where things get really interesting for your sales team.
Instead of just seeing a number (Lead Score: 87… out of what? Based on what?), reps now see two custom fields on the lead record:
That’s a world of difference. A sales rep reading that reason knows exactly why this lead matters and how to approach the conversation. They can reference the webinar, mention the ROI calculator, and speak to the specific use case that likely drove the interest.
You might be wondering: why go through the trouble of fine-tuning? Can’t you just send lead data to GPT-4 with a good prompt?
You could, and it would probably be better than rules-based scoring. But a fine-tuned model has some significant advantages:
It’s calibrated to your business. A generic model doesn’t know that leads from the financial services industry close 3x faster for your company, or that Directors convert better than VPs in your space. Your fine-tuned model learned these patterns from your actual data.
It’s more consistent. Fine-tuned models produce more predictable outputs because they’ve internalized your scoring rubric. A generic model with a long prompt can drift or interpret things differently across calls.
It’s faster and cheaper. Fine-tuned models on smaller base models (like gpt-4o-mini) are significantly cheaper per call than sending large prompts to a full-size model. When you’re scoring leads in real time, cost per API call matters.
It improves over time. Every quarter, you can pull new closed-won and closed-lost data, add it to your training set, and re-fine-tune. The model gets smarter as your business evolves.
One last thing worth mentioning. Your fine-tuned model is only as good as the data it was trained on. Markets shift, your product evolves, and the profile of your best customers can change.
We recommend re-training your model every quarter. Pull the latest 3-6 months of closed deals, re-grade them, and run another fine-tuning job. It’s not a huge lift once you have the pipeline set up, and it ensures your scoring stays relevant.
You can even compare model versions by running both the old and new models on the same set of recent leads and seeing which one better predicts actual outcomes. Think of it as A/B testing for your scoring model.
Traditional lead scoring served us well for a long time. But buyer behavior has gotten more complex, and a rules engine that adds up points just can’t keep pace.
By fine-tuning an AI model on your own historical data, you get a scoring system that actually understands what a good lead looks like for your specific business.
And by deploying it through an iPaaS solution connected to Marketo, you can score leads in real time, complete with a human-readable explanation that sales teams will actually trust and use.
If you want help setting up AI-powered lead scoring for your Marketo instance, or if you’re curious about what else AI can do for your marketing ops, reach out to us here.
We’d love to help you get started.