Events in: a webhook source, its table, and an automation that reads it

What you will take away from this lesson

Lesson 604 named three ways a run is asked for and mentioned a fourth: events another system sends. A point-of-sale system posts each sale, a CRM posts each contact change, an email platform posts each bounce. Plexara gives each of those senders an address of its own, checks that every request came from it, stores the events, and serves them as a table an automation reads like any other.

This lesson follows one feed on the demo tenant from the sender to the rows an automation acts on: the source an administrator created, what the sender is told, where the events land, how they are kept, how an administrator watches them, and an automation that picks up only what is new on every run.

Learning Objectives

01Say who creates a webhook source, where, and which of its choices are fixed once it exists.
02Choose how a sender proves itself, rotate its secret without dropping an event, and explain why a timestamp makes a replayed request fail.
03Read what a sender is told for each request, in the order the checks run, and why an acknowledgement means the event is stored.
04Query a source’s events as one table from the moment they are acknowledged, and take fields out of the payload.
05Explain compaction and retention as the source’s page shows them, and find a compacted window by search.
06Read the Webhooks page: request volume, each source’s health, and the rejected requests.
07Write an automation that reads only what is new with a watermark, and neither misses nor double-counts an event.

Where this lesson sits

The 606 weekly review ran on scripts that went and fetched their data on a schedule. This lesson adds the other direction: data another system pushes to Plexara as it happens, and the automations that read it. It uses the script loop from 602, the outputs from 603, and the schedules from 604.

600 Series: Automations

Open index

The 600 series picks up the registered table from the 500 series and the prompt mechanics from the 400 series. It is the last numbered series in the curriculum.

The feed on the demo tenant

Every figure in this lesson comes from one feed on the ACME demo tenant: simulated point-of-sale traffic drawn from the real stores and product catalog, posted to a webhook source every five minutes, and read by two automations that read it in two different ways.

The feed on the demo tenant

The sender ACME POS sales simulator: An automation that runs every five minutes, draws a batch of 10 to 50 sales from the real ACME stores and product catalog, and posts the batch to the source in one request through an API connection that holds the source’s token. It stands in for the point-of-sale system a retailer would point at Plexara.
The source pos-sales: Created by an administrator under Admin > Webhooks. It splits each batch into one event per sale, takes the sale id as the event id, sale.completed or sale.returned as the event type, and the store id as the key.
The table scratch.uploads.webhook_pos_sales: One row per sale, queryable as soon as the batch is acknowledged. When this lesson was written it held 58,588 events from all 303 stores, 2,313 of them returns, with 58,588 distinct event ids.
A reader that recomputes ACME Live Sales dashboard (refresh): Every fifteen minutes it reads the 24 hours before its fire time and refreshes the data region of a sales dashboard: revenue by region, by hour, by payment method, and the latest sales.
A reader that keeps its place ACME POS returns watch (lesson 607): Written for this lesson. Each run reads only the events that arrived since the last one and files the returns among them as a CSV feed.

The receiving side knows nothing about its readers. The source receives, stores, compacts, and applies retention whether anything reads its table or not, and readers come and go without the source changing: a dashboard that recomputes a day, an automation that picks up where it left off, an analyst asking the agent a question.

On the Automations page, the Schedules tab draws every cadence on one chart, one row per script with a mark at each fire: the simulator’s row carries a mark every five minutes, the dashboard’s one every fifteen.

Creating a source

A source is created by an administrator on your team, in the portal under Admin > Webhooks. The form has four groups, and a field left empty takes its standard value.

Admin > Webhooks > New webhook source

1
Source
A name, the connection whose scratch catalog will hold the table, and who sees the compacted files. The name becomes the sender’s address, /hooks/<name>, and the table, webhook_<name> with each hyphen as an underscore.
2
Authentication
How a request proves it came from the sender, and the secret it proves it with. The secret is never shown again once it is saved.
3
Events
Where in the body the events are when a sender batches them, and which fields are the event id, the event type, and the key. Duplicates are removed on the event id; with none, on a hash of the event. The key is the entity an event is about, so a reader can keep the latest event per key.
4
Limits and retention
How often events are compacted, the largest body and the most events a request may carry, an optional rate, and how long raw events and compacted files are kept.

Creating a source is an administrator’s action in the portal. The agent has no tool for it, so a source exists because a person on your team decided it should. The name, the connection, and who sees the compacted files are fixed at creation: the table was created under that name on that connection, and the windows already written sit in that library. Everything else can be edited on the source’s page, and the source can be disabled without being deleted. The demo’s source checks a token in a header, keeps its compacted files visible to administrators, and compacts every hour.

How a sender proves itself

Every request has to show it came from the sender the source was made for. The source accepts one kind of proof, chosen to match what the sending system can produce.

How a request proves it came from the sender

HMAC signature
A signature of the body in a header
SHA-256, SHA-1, or SHA-512, in hex or base64, with an optional prefix such as sha256=. The signed content can be the body alone, a timestamp and the body, or an id, a timestamp, and the body as Standard Webhooks signs them. A Stripe-style header, with the timestamp inside the signature header, is a choice of its own. A header carrying several signatures verifies when one of them matches.
Token in a header
The secret itself, in a named header
The simplest proof for a sender you configure yourself. This is what the demo’s pos-sales source uses.
Basic credentials
A username and the secret as the password
For a sender whose webhook settings offer only a username and a password.
Token in the URL
The secret as one more path segment
For a sender that can set nothing but a URL. The portal warns that the secret then appears in every log that records addresses, so it is the last choice, not the first.

One mode per source. With a timestamp in the signature, a request whose timestamp is more than the tolerance away from the platform’s clock is refused even when its signature is right, which is what makes a captured request useless to replay. Secrets are stored encrypted and never shown again; the source’s page says only whether one is set. Rotating a secret takes a new one and a number of hours to keep accepting the previous one, so the sender can be moved over with no window in which its requests fail. A source can also answer the CloudEvents handshake for a sender that asks permission before it delivers.

What a sender is told

A webhook sender acts on the status code it gets back. Each request is checked in a fixed order, and the first check that fails decides the answer.

What a sender is told, in the order the checks run

CheckAnswer when it fails

The source exists and is enabled: 404, the same answer for a disabled source and a name that is not a source
The body is no larger than the source allows: 413, nothing stored
The request authenticates: 401, nothing stored
The source’s rate, when it has one: 429, with when to retry
The body is JSON, and the batch path names an array: 400, nothing stored
The request holds no more events than the source can buffer: 413; a batch that can never fit is not worth retrying
The events fit in the buffer now: 503, retry in a second
The events are written to storage: 503, retry in five seconds
Everything above passed: 202, with how many events were accepted

The first refusal ends the request. A 202 means the events are in storage, not in a queue that might lose them. Delivery is at least once: a sender that did not see its acknowledgement sends the request again, and the copy is removed when its window is compacted. Backpressure is the sender’s own retry queue, told when to come back; Plexara keeps no backlog of its own beyond each source’s buffer.

One delivery

The demo’s sender is itself an automation, so its record shows both sides of a delivery: what it sent and what the source told it.

One delivery from the demo’s sender

What the agent calls. This is the exchange the agent has with Plexara on your behalf, shown for the technical reader. You ask in plain language; you never type any of it.

manage_script command=get_run
  name    acme-pos-sales-simulator
  run_id  dpx_2256ee0d…

-> status          succeeded
   trigger         schedule
   duration_ms     1572
   params          {"min_sales": 10, "max_sales": 50}
   result          {"sales": 47, "returns": 3, "accepted": 47}

The simulator posted 47 sales in one request, the source split them into 47 events and answered 202 with {"accepted": 47}, and the run copied that count into its result. When the answer is anything but 202 the simulator fails its run as retryable, so a dropped batch shows on the Automations page as a failed run instead of a quiet gap in the table. A real sender behaves the same way through its own retry queue.

Where the events land

Readers name one table per source, webhook_ followed by the source’s name, in the scratch catalog of the source’s connection. Each row is one event.

webhook_pos_sales: the columns a reader uses

received_at: When the platform received the request, in UTC.
landed_at: When the stored batch holding the event was written. The column to keep a place by.
event_id: The sender’s id for the event; the sale id on the demo.
event_type: sale.completed or sale.returned on the demo.
key: The entity the event is about; the store id on the demo.
payload: The event’s JSON as the sender sent it.
dt, hour, minute: The day and the window the event was received in, for narrowing a query.

The schema is fixed, so a sender that adds a field breaks nothing: a reader takes what it needs with json_extract_scalar(payload, '$.field'). The table also carries a content hash and the receiving replica. It is listed under Scratch Tables like any registered table, is removed only with its source, and is queried by whoever is granted its connection.

The table in the portal

The source’s table is listed in Scratch Tables, where its page gives a sample query, the columns, and the source behind it.

The Scratch Tables page for scratch.uploads.webhook_pos_sales: a sample query selecting received_at, event_id, event_type, key and payload for today’s partition newest first, eleven columns (received_at, landed_at, event_id, event_type, key, content_hash, replica, payload, dt, hour, minute), and the pos-sales webhook source behind the table. — The pos-sales table in Scratch Tables, beside the spreadsheets people register: a sample query for today’s events, its columns, and a link to the source.

Reading the table

Nothing about the table is special to the agent or to a script. It is queried with the same tool as any other, by anyone granted the connection.

The whole feed, read as one table

What the agent calls. This is the exchange the agent has with Plexara on your behalf, shown for the technical reader. You ask in plain language; you never type any of it.

trino_query
  connection  acme
  sql   SELECT count(*) AS events, count(DISTINCT event_id) AS ids,
               count_if(event_type = 'sale.returned') AS returns,
               count(DISTINCT key) AS stores, count(DISTINCT dt) AS days
        FROM scratch.uploads.webhook_pos_sales

-> events   58588
   ids      58588
   returns  2313
   stores   303
   days     7

The table is a view over two kinds of storage: the compacted window files, and the raw batches of the window still open. For each window it reads the compacted file once there is one and the raw batches until then, so an event is queryable the moment its 202 is sent and no window is missing or served twice. The query does not change when a window is compacted underneath it.

Compaction and retention

Raw batches are written as events arrive, a few per window. Compaction turns each finished window into one tidy file, and retention decides how long either is kept. Both are shown on the source’s page.

Compaction and retention on the pos-sales source’s page

Every hour: Compacted every: one Parquet file per window
7 days: Raw events kept after compaction
90 days: Compacted files kept
163: Compacted windows in the library when this lesson was written

A window is compacted shortly after it ends: every batch written for it is read, one event is kept per event id (the earliest received), and the result is written as one Parquet file. The window length is chosen per source from a fixed set that divides the hour: 1, 2, 3, 4, 5, 6, 10, 12, 15, 20, 30, or 60 minutes. A shorter window removes duplicates sooner and produces more files; every event is queryable on acknowledgement either way. Retention never deletes an event that has not reached a compacted file, and the page shows the oldest window still held beside both periods, so a team with personal data in its payloads can see exactly how far back the source reaches.

The compacted files

A compacted window is a file in the resource library, which is how the events can be handed to somebody or to another tool as a file instead of a query.

A compacted window, found by search

What the agent calls. This is the exchange the agent has with Plexara on your behalf, shown for the technical reader. You ask in plain language; you never type any of it.

search
  intent   pos-sales webhook compaction window parquet
  sources  ["resources"]

-> pos-sales 2026-10-01 22:00 UTC
     mcp://persona/admin/webhooks/pos-sales/2026-10-01/22-00.parquet
     Webhook events received by source pos-sales in the compaction
     window starting 2026-10-01 22:00 UTC.
     table  scratch.uploads.webhook_pos_sales  (parquet)
   pos-sales 2026-10-05 22:00 UTC
     mcp://persona/admin/webhooks/pos-sales/2026-10-05/22-00.parquet
   …

Each compacted window is a resource in the library of the persona chosen when the source was created, filed under the source’s name and the day, and named by the hour and minute the window starts. It opens in the portal’s Parquet viewer, carries the source’s table with it, and is found by search like any resource the reader can see. Its contents are not indexed for search; the table is how the events are read.

The Webhooks page

An administrator watches every source from one page under Admin > Webhooks. It answers the questions that matter about a feed: is it arriving, is anything being refused, and is anything not being kept.

Admin > Webhooks

Requests: Request volume by outcome, stacked, for every source or one, over the last hour or the last 24 hours, with the total of each outcome beneath. On the demo, pos-sales showed 288 requests accepted in the last day and none rejected: one every five minutes.
Sources: One row per source: its health, when it last received an event, requests accepted and rejected in the last hour and the last day, windows waiting to be compacted, and windows whose compaction failed. A row opens the source’s page.
Recent rejections: Every rejected request across the sources, grouped by outcome (Unauthorized, Too large, Rate limited, Invalid body, Buffer full, Write failed), each with its source, time, and reason, such as “the signature does not match”. A burst of the same refusal is one row with a count. The body of a rejected request is never stored.

A source’s health

Disabled: An administrator turned the source off.
Failing: One or more of its windows failed their last compaction; the page shows the newest error.
Silent: Enabled, and no event received in the last 24 hours. A sender that stops produces no error on the receiving side, so this is how a stopped sender shows.
Receiving: Enabled, an event received in the last 24 hours, nothing failing.

When more than one applies, the first in that order is shown: a disabled source receives nothing by choice, and a failing one is losing what it did receive, which matters more than silence. Rejections are kept per outcome, so a burst of rate-limited requests never pushes out the unauthorized rows that would show someone sending forged requests. The source’s own page repeats its share of all this beside the address to give the sender and the table readers query.

An automation that reads only what is new

The pattern for work done once per event is a watermark kept in the script’s state. The script written for this lesson files the returns among each run’s new sales.

acme-pos-returns-watch-607, its main()

What the agent calls. This is the exchange the agent has with Plexara on your behalf, shown for the technical reader. You ask in plain language; you never type any of it.

def main():
    since = run.state.get("landed_through", "")
    seen = run.state.get("recent_ids", [])
    rows = read_events(since)
    new = [r for r in rows if r["event_id"] not in seen]
    returns = returns_of(new)
    if returns:
        platform.export("acme-pos-returns-607", returns, format = "csv")
    if rows:
        platform.save_state({
            "landed_through": rows[-1]["landed_at"],
            "recent_ids": [r["event_id"] for r in rows if r["in_overlap"]],
        })
    platform.result({...})

# read_events selects events whose landed_at is after the watermark,
# starting five minutes before it, up to the run's fire time.
# With no watermark yet, it reads the hour before the fire time.

The agent wrote the script in one session through the loop from 602: a recorded draft against the live feed, three tests (a first run with no watermark, a run that re-reads an event it already has, a run that finds nothing), and a save that reached every statement. It is set to run one at a time, because a script that moves a watermark should never have two runs reading the same place.

Nobody draws the script’s diagram. The Flow tab on its page reads it off the saved source: the state read at the top, one card for the query, a decision on whether there are returns to file with the export on its yes arm, and a second on whether anything was read with the state saved on its yes arm. With a run selected, each card says what it did in that run, and Timeline places the run’s calls in time.

Three runs of it

The script was run three times on the demo tenant, a minute or two apart, around one of the simulator’s deliveries.

Three runs, and where each one started

What the agent calls. This is the exchange the agent has with Plexara on your behalf, shown for the technical reader. You ask in plain language; you never type any of it.

run_script  name acme-pos-returns-watch-607

1  state_read     {}
   result         {"read": 313, "already_seen": 0, "new_events": 313, "new_returns": 14}
   outputs        acme-pos-returns-607  version 1  (csv, 14 rows)
   state_written  {"landed_through": "2026-10-07 19:20:08.455789",
                   "recent_ids": [11 ids from the batch that landed last]}

2  state_read     {"landed_through": "… 19:20:08.455789", "recent_ids": [11 ids]}
   result         {"read": 11, "already_seen": 11, "new_events": 0, "new_returns": 0}
   outputs        none

3  state_read     {"landed_through": "… 19:20:08.455789", "recent_ids": [11 ids]}
   result         {"read": 58, "already_seen": 11, "new_events": 47, "new_returns": 3}
   outputs        acme-pos-returns-607  version 2  (csv, 3 rows)
   state_written  {"landed_through": "… 19:25:08.417310", "recent_ids": [58 ids]}

The first run had no watermark, read the hour before it, and filed 14 returns. The second ran before the next batch arrived: it re-read the eleven events inside its five-minute overlap, recognized every one, and wrote nothing. The third ran after the simulator’s next delivery: it re-read the same eleven, skipped them, and took the 47 new sales, which is exactly the count the source accepted from that delivery. Three of them were returns, so the feed got a second version with three rows.

The rules the watermark follows

Six rules make the difference between an automation that reads a feed correctly and one that is right most days.

Reading only what is new, without missing or repeating

01
Keep the place by landed_at, not received_at
A batch written late carries received_at values earlier than events already read, so a watermark on received_at can step over it. landed_at is when the batch reached storage, which is the order a reader sees them in.
02
Start a little before the watermark
The overlap covers small clock differences between the replicas that receive events and a batch written a moment late. The script reads five minutes back.
03
Remember the ids inside the overlap
An event read twice is harmless when the script knows it already has it. Keeping only the ids from the last five minutes of the previous run keeps the state small however long the feed runs.
04
Key on the event id until the window is compacted
A sender that retries produces a duplicate, and the copy is removed only when its window is compacted. Anything that must not double-count, a count, a sum, a message sent per event, keys on event_id.
05
Let a failed run leave the watermark alone
State is saved only when a run succeeds, so a run that fails halfway leaves the place where it was and the next run reads the same events again.
06
Run one at a time
A script that moves a watermark is set to run one at a time, so a run asked for by hand is refused while a scheduled one is open, and a fire that arrives during a run is recorded as skipped.

Recomputing a window instead

Events in, end to end

The point-of-sale feed on the demo tenant arrives every five minutes, is verified, stored, and acknowledged, is queryable at once, is compacted every hour, and is read by a dashboard and by an automation that takes each sale once.

Key terms

Five terms cover how an event arrives and how an automation reads it.

Key Terms

Webhook source: An address another system posts events to, created by an administrator under Admin > Webhooks, with its own proof of sender, its own table, and its own compaction and retention.
Acknowledgement202: The answer a sender gets once its events are written to storage. Anything not acknowledged is the sender’s to send again, which is why delivery is at least once.
Compacted window: One period of a source’s events, de-duplicated on event id and written as one Parquet file in a persona’s library, found by search and opened in the Parquet viewer.
Event id: The sender’s id for an event, or a hash of the event when the sender gives none. Compaction keeps one event per id, and a reader that must not double-count keys on it.
Watermark: The place an automation keeps in its state between runs, here the landed_at of the newest event it read. Saved only when a run succeeds, so a failed run is read again.

607 - Events in: a webhook source, its table, and an automation that reads it

What you will take away from this lesson

Where this lesson sits

The feed on the demo tenant

Creating a source

How a sender proves itself

What a sender is told

One delivery

Where the events land

The table in the portal

Reading the table

Compaction and retention

The compacted files

The Webhooks page

An automation that reads only what is new

Three runs of it

The rules the watermark follows

Recomputing a window instead

Events in, end to end

Key terms

Related reading

102 - Tokens and your budget

105 - What is an AI agent?

202 - Your first day with Plexara

Cookie Preferences