Tutorials / Databases & encryption

Tutorial · Node.js or Python · 20 minutes

Extract data from email and save it to a database, encrypted

Supplier invoices arrive by email. By the end of this tutorial, each one becomes a row in your database: the invoice number, amount and due date ready for reports, and the supplier's details and the email itself encrypted, so a leaked backup or a curious query shows nothing personal.

Extract Data from Email & Save It to a Database, Encrypted (Node.js) · Watch on YouTube

Email arrivesat invoices@yourcompany.com
GhostParse parses ita parser pulls out the fields
Signed webhookJSON to your app
Your app encryptsAES-256-GCM, per field
Database rowsearchable, never readable

Before you start

A free GhostParse account (sign up) with a domain, or your instant inbox address.
Node.js 22.13 or later, or Python 3.9 or later. No database server: both versions use SQLite, built in.
The finished code is in examples/ of our repository; everything below is the whole of it.

1. Create a parser for the invoices

A parser turns the email's text into named, typed fields, so your code never has to. In the dashboard, open Parsers → New parser, call it Supplier invoices, choose Rules and add three fields:

Field	Type	Rule
`invoice_number`	Text	Text after `Invoice number:`
`amount_due`	Number	Text after `Amount due:` (£1,284.60 becomes 1284.6)
`due_date`	Date	Text after `Due date:` (14 November 2026 becomes 2026-11-14)

Then, under Domains, add the address invoices@ and choose this parser for it. Every email to that address now arrives with an extracted object:

{
  "type": "email.received",
  "data": {
    "from": { "address": "accounts@lumen-supplies.example", "name": "Lumen Office Supplies" },
    "subject": "Invoice INV-20977",
    "text": "Hello,\n\nInvoice number: INV-20977\nAmount due: £312.00\nDue date: 14 November 2026\n…",
    "extracted": {
      "parser": "Supplier invoices",
      "method": "rules",
      "fields": { "invoice_number": "INV-20977", "amount_due": 312, "due_date": "2026-11-14" },
      "error": null
    },
    …
  }
}

Suppliers all write invoices differently? Choose AI instead of Rules (or start from the ready-made Supplier invoices parser): you describe the fields and it finds them in any layout. The rest of this tutorial doesn't change.

2. Decide what to encrypt

Encrypt what would hurt if it leaked; leave readable what you need to add up, sort and filter in SQL.

Column	Stored as	Why
`invoice_number`, `amount_due`, `due_date`	Plain	Not personal, and you'll want `SUM()` and `ORDER BY`
`supplier_email_enc`, `supplier_name_enc`	Encrypted	Personal data about a person or business
`body_enc`	Encrypted	The email can contain anything: bank details, names, addresses
`supplier_email_idx`	Blind index	Lets you find a supplier's invoices without decrypting anything
`webhook_id`	Primary key	A retried delivery has the same id, so it can't become a duplicate row

// SQLite with Node's built-in driver. Any SQL database works the same way: the encrypted columns are just text.
import { DatabaseSync } from "node:sqlite";

export function openDatabase(path = process.env.DATABASE_PATH ?? "invoices.db") {
  const db = new DatabaseSync(path);
  db.exec(`CREATE TABLE IF NOT EXISTS invoices (
    webhook_id         TEXT PRIMARY KEY,  -- the webhook-id header: a retried delivery is the same row, not a duplicate
    received_at        TEXT NOT NULL,
    invoice_number     TEXT,              -- not sensitive, kept readable for reports and search
    amount_due         REAL,
    due_date           TEXT,
    supplier_email_enc TEXT,              -- encrypted (AES-256-GCM)
    supplier_email_idx TEXT,              -- blind index: find a supplier's invoices without decrypting
    supplier_name_enc  TEXT,              -- encrypted
    body_enc           TEXT               -- the email's text, encrypted
  )`);
  db.exec("CREATE INDEX IF NOT EXISTS invoices_supplier ON invoices (supplier_email_idx)");
  return db;
}

"""SQLite from the standard library. Any SQL database works the same way: the encrypted columns are just text."""
import os
import sqlite3


def open_database(path=None):
    db = sqlite3.connect(path or os.environ.get("DATABASE_PATH", "invoices.db"))
    db.row_factory = sqlite3.Row
    db.executescript("""
      CREATE TABLE IF NOT EXISTS invoices (
        webhook_id         TEXT PRIMARY KEY,  -- the webhook-id header: a retried delivery is the same row
        received_at        TEXT NOT NULL,
        invoice_number     TEXT,              -- not sensitive, kept readable for reports and search
        amount_due         REAL,
        due_date           TEXT,
        supplier_email_enc TEXT,              -- encrypted (AES-256-GCM)
        supplier_email_idx TEXT,              -- blind index: find a supplier's invoices without decrypting
        supplier_name_enc  TEXT,
        body_enc           TEXT
      );
      CREATE INDEX IF NOT EXISTS invoices_supplier ON invoices (supplier_email_idx);
    """)
    return db

3. Field-level encryption

One module, no dependencies beyond the standard crypto library. Read it once: every line matters.

// Field-level encryption for the database: AES-256-GCM for values you need to read back, and a keyed hash (a "blind
// index") for values you need to look up without decrypting. Both keys come from one master key in DATA_KEY.
import { createCipheriv, createDecipheriv, createHmac, hkdfSync, randomBytes } from "node:crypto";

function masterKey() {
  const key = Buffer.from(process.env.DATA_KEY ?? "", "base64");
  if (key.length !== 32) throw new Error("Set DATA_KEY to 32 random bytes, base64 (npm run key makes one)");
  return key;
}

// Separate keys for separate jobs, derived from the master key: rotating or leaking one use never weakens the other.
const derive = (purpose) => Buffer.from(hkdfSync("sha256", masterKey(), Buffer.alloc(0), `invoices-tutorial:${purpose}`, 32));
const ENC_KEY = derive("encrypt");
const INDEX_KEY = derive("index");

/**
 * Encrypt a value. `context` (e.g. "supplier_email") is authenticated but not stored: a ciphertext copied into
 * another column won't decrypt there. Output: "v1." + base64(iv | ciphertext | tag), the same layout as the
 * Python version of this tutorial, so either can read the other's data.
 */
export function encrypt(plaintext, context) {
  if (plaintext === null || plaintext === undefined) return null;
  const iv = randomBytes(12); // never reuse an IV with the same key: a fresh random one each time
  const cipher = createCipheriv("aes-256-gcm", ENC_KEY, iv);
  cipher.setAAD(Buffer.from(context));
  const body = Buffer.concat([cipher.update(String(plaintext), "utf8"), cipher.final()]);
  return `v1.${Buffer.concat([iv, body, cipher.getAuthTag()]).toString("base64")}`;
}

export function decrypt(stored, context) {
  if (stored === null || stored === undefined) return null;
  if (!stored.startsWith("v1.")) throw new Error("Unknown ciphertext version");
  const raw = Buffer.from(stored.slice(3), "base64");
  const decipher = createDecipheriv("aes-256-gcm", ENC_KEY, raw.subarray(0, 12));
  decipher.setAAD(Buffer.from(context));
  decipher.setAuthTag(raw.subarray(raw.length - 16));
  return Buffer.concat([decipher.update(raw.subarray(12, raw.length - 16)), decipher.final()]).toString("utf8"); // throws if tampered with
}

/** The same input always gives the same index, so you can search on it; without INDEX_KEY it reveals nothing. */
export function blindIndex(value) {
  return createHmac("sha256", INDEX_KEY).update(String(value).trim().toLowerCase()).digest("hex");
}

"""Field-level encryption for the database: AES-256-GCM for values you need to read back, and a keyed hash (a
"blind index") for values you need to look up without decrypting. Both keys come from one master key in DATA_KEY."""
import base64
import hashlib
import hmac
import os

from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
from cryptography.hazmat.primitives.kdf.hkdf import HKDF


def _master_key() -> bytes:
    key = base64.b64decode(os.environ.get("DATA_KEY", ""))
    if len(key) != 32:
        raise RuntimeError("Set DATA_KEY to 32 random bytes, base64: python -c \"import os,base64;print(base64.b64encode(os.urandom(32)).decode())\"")
    return key


def _derive(purpose: str) -> bytes:
    # Separate keys for separate jobs, derived from the master key.
    return HKDF(algorithm=hashes.SHA256(), length=32, salt=None, info=f"invoices-tutorial:{purpose}".encode()).derive(_master_key())


_ENC_KEY = _derive("encrypt")
_INDEX_KEY = _derive("index")


def encrypt(plaintext, context: str):
    """`context` (e.g. "supplier_email") is authenticated but not stored. Output: "v1." + base64(iv | ciphertext+tag)."""
    if plaintext is None:
        return None
    iv = os.urandom(12)  # a fresh random IV every time: never reuse one with the same key
    return "v1." + base64.b64encode(iv + AESGCM(_ENC_KEY).encrypt(iv, str(plaintext).encode(), context.encode())).decode()


def decrypt(stored, context: str):
    if stored is None:
        return None
    if not stored.startswith("v1."):
        raise ValueError("Unknown ciphertext version")
    raw = base64.b64decode(stored[3:])
    return AESGCM(_ENC_KEY).decrypt(raw[:12], raw[12:], context.encode()).decode()  # raises if tampered with


def blind_index(value: str) -> str:
    """The same input always gives the same index, so you can search on it; without the key it reveals nothing."""
    return hmac.new(_INDEX_KEY, value.strip().lower().encode(), hashlib.sha256).hexdigest()

AES-256-GCM encrypts and authenticates: a changed byte doesn't decrypt to rubbish, it fails loudly.
A new random IV every time, stored with the ciphertext. Encrypting the same email twice gives two different values, so nobody can tell two rows hold the same supplier.
The column name is the "associated data". It isn't stored but is checked on decrypt, so a ciphertext copied from body_enc into supplier_name_enc won't decrypt.
One master key, two derived keys (HKDF): one for encrypting, one for the blind index. The index key can't decrypt anything.
The blind index is an HMAC of the lowercased, trimmed email: the same supplier always gets the same index, so WHERE supplier_email_idx = ? works, yet without the key it reveals nothing.
The v1. prefix leaves room to change algorithm or key later and still read old rows.

The Node.js and Python versions write exactly the same format, so you can mix them: one saving, the other reporting.

4. Receive the webhook and save the invoice

The server checks every webhook's signature with the GhostParse SDK before it trusts a single field, then saves the row.

npm install @wildye/email-parser

pip install wildye-email-parser cryptography

// Receives GhostParse webhooks, checks they're genuine, and saves each invoice with its sensitive fields encrypted.
//   GHOSTPARSE_WEBHOOK_SECRET=whsec_… DATA_KEY=… node server.mjs
import { createServer } from "node:http";
import { verifyWebhook, WebhookVerificationError } from "@wildye/email-parser";
import { openDatabase } from "./database.mjs";
import { blindIndex, encrypt } from "./encryption.mjs";

const SECRET = process.env.GHOSTPARSE_WEBHOOK_SECRET;
if (!SECRET) throw new Error("Set GHOSTPARSE_WEBHOOK_SECRET (Dashboard → Inbound webhook → Signing secret)");
const db = openDatabase();
const insert = db.prepare(`INSERT OR IGNORE INTO invoices
  (webhook_id, received_at, invoice_number, amount_due, due_date, supplier_email_enc, supplier_email_idx, supplier_name_enc, body_enc)
  VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`);

function save(webhookId, email) {
  const f = email.extracted?.fields ?? {};
  const supplier = email.from?.address ?? null;
  const result = insert.run(
    webhookId,
    email.received_at ?? new Date().toISOString(),
    f.invoice_number ?? null,
    typeof f.amount_due === "number" ? f.amount_due : null,
    f.due_date ?? null,
    encrypt(supplier, "supplier_email"),
    supplier ? blindIndex(supplier) : null,
    encrypt(email.from?.name || null, "supplier_name"),
    encrypt(email.text ?? null, "body"),
  );
  return result.changes === 1;
}

createServer((req, res) => {
  if (req.method !== "POST" || req.url !== "/webhooks/email") return res.writeHead(404).end();
  const chunks = [];
  req.on("data", (c) => chunks.push(c));
  req.on("end", () => {
    try {
      // Verify the raw body exactly as received: re-serialised JSON wouldn't match the signature.
      const event = verifyWebhook(Buffer.concat(chunks), req.headers, SECRET);
      if (event.type === "email.received") {
        const stored = save(req.headers["webhook-id"], event.data);
        console.log(`${stored ? "saved" : "already had"} ${event.data.extracted?.fields?.invoice_number ?? "(no invoice number)"} from ${event.data.from?.address}`);
      }
      res.writeHead(200).end("ok");
    } catch (err) {
      if (err instanceof WebhookVerificationError) {
        console.warn(`refused: ${err.message}`);
        return res.writeHead(401).end("bad signature");
      }
      console.error(err);
      res.writeHead(500).end(); // GhostParse retries: nothing is lost
    }
  });
}).listen(Number(process.env.PORT ?? 3000), () => console.log(`Listening on http://localhost:${process.env.PORT ?? 3000}/webhooks/email`));

"""Receives GhostParse webhooks, checks they're genuine, and saves each invoice with its sensitive fields encrypted.
    GHOSTPARSE_WEBHOOK_SECRET=whsec_… DATA_KEY=… python server.py"""
import os
from datetime import datetime, timezone
from http.server import BaseHTTPRequestHandler, HTTPServer

from email_parser import WebhookVerificationError, verify_webhook

from database import open_database
from encryption import blind_index, encrypt

SECRET = os.environ.get("GHOSTPARSE_WEBHOOK_SECRET") or exit("Set GHOSTPARSE_WEBHOOK_SECRET (Dashboard → Inbound webhook)")
db = open_database()


def save(webhook_id: str, email) -> bool:
    fields = (email.raw.get("extracted") or {}).get("fields") or {}
    supplier = email.from_address
    cur = db.execute(
        """INSERT OR IGNORE INTO invoices (webhook_id, received_at, invoice_number, amount_due, due_date,
             supplier_email_enc, supplier_email_idx, supplier_name_enc, body_enc) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)""",
        (
            webhook_id,
            email.raw.get("received_at") or datetime.now(timezone.utc).isoformat(),
            fields.get("invoice_number"),
            fields.get("amount_due") if isinstance(fields.get("amount_due"), (int, float)) else None,
            fields.get("due_date"),
            encrypt(supplier, "supplier_email"),
            blind_index(supplier) if supplier else None,
            encrypt((email.raw.get("from") or {}).get("name") or None, "supplier_name"),
            encrypt(email.raw.get("text"), "body"),
        ),
    )
    db.commit()
    return cur.rowcount == 1


class Handler(BaseHTTPRequestHandler):
    def do_POST(self):
        if self.path != "/webhooks/email":
            return self._reply(404, b"")
        body = self.rfile.read(int(self.headers.get("content-length", 0)))
        try:
            # Verify the raw body exactly as received.
            event = verify_webhook(body, self.headers, SECRET)
        except WebhookVerificationError as err:
            print(f"refused: {err}", flush=True)
            return self._reply(401, b"bad signature")
        if event.type == "email.received":
            email = event.email
            stored = save(self.headers["webhook-id"], email)
            print(f"{'saved' if stored else 'already had'} {email.field('invoice_number') or '(no invoice number)'} from {email.from_address}", flush=True)
        self._reply(200, b"ok")

    def _reply(self, status, body):
        self.send_response(status)
        self.end_headers()
        self.wfile.write(body)

    def log_message(self, *args):
        pass


if __name__ == "__main__":
    port = int(os.environ.get("PORT", 3000))
    print(f"Listening on http://localhost:{port}/webhooks/email", flush=True)
    HTTPServer(("", port), Handler).serve_forever()

Verify the raw body, exactly as received. Parsing the JSON and serialising it again would change the bytes and fail the check.
401 for a bad signature: anyone can find your URL, but only GhostParse has the signing secret.
INSERT OR IGNORE on the webhook id: if your app answers slowly and we retry, the second delivery changes nothing.
500 on an error: we retry with backoff, so a database hiccup doesn't lose an invoice.

5. Run it and send a test invoice

Make a key and start the server. Keep the key somewhere safe: without it, the encrypted columns can never be read again.

export DATA_KEY=$(npm run -s key)
export GHOSTPARSE_WEBHOOK_SECRET=whsec_…   # Dashboard → Inbound webhook → Signing secret
node server.mjs

export DATA_KEY=$(python -c "import os,base64;print(base64.b64encode(os.urandom(32)).decode())")
export GHOSTPARSE_WEBHOOK_SECRET=whsec_…   # Dashboard → Inbound webhook → Signing secret
python server.py

You don't need to send a real email to try it. The CLI's sandbox runs an .eml file through your actual parser and posts the signed webhook to your laptop (nothing is delivered, stored or counted):

export EMAIL_PARSER_API_KEY=ek_live_…   # Dashboard → API keys
npx @wildye/email-parser-cli simulate http://localhost:3000/webhooks/email \
  --eml invoice.eml --parser prs_… --secret $GHOSTPARSE_WEBHOOK_SECRET

Listening on http://localhost:3000/webhooks/email
saved INV-20977 from accounts@lumen-supplies.example

Run the same command again: the server says already had INV-20977, because the webhook id is the same. Add --bad-signature to check forged webhooks are refused.

6. Read it back

With the key, everything decrypts:

node show.mjs

python show.py

┌─────────┬─────────────┬────────┬──────────────┬─────────────────────────┬───────────────────────────────────┐
│ (index) │ invoice     │ amount │ due          │ supplier                │ email                             │
├─────────┼─────────────┼────────┼──────────────┼─────────────────────────┼───────────────────────────────────┤
│ 0       │ 'INV-20977' │ 312    │ '2026-11-14' │ 'Lumen Office Supplies' │ 'accounts@lumen-supplies.example' │
│ 1       │ 'INV-20981' │ 1284.6 │ '2026-11-14' │ 'Lumen Office Supplies' │ 'accounts@lumen-supplies.example' │
└─────────┴─────────────┴────────┴──────────────┴─────────────────────────┴───────────────────────────────────┘

What's actually in the database (and in every backup of it) is ciphertext:

node show.mjs --raw

python show.py --raw

┌─────────┬─────────────┬─────────────────────────────────┬─────────────────────────────────┐
│ (index) │ invoice     │ supplier_email_enc              │ body_enc                        │
├─────────┼─────────────┼─────────────────────────────────┼─────────────────────────────────┤
│ 0       │ 'INV-20977' │ 'v1.MkyY8GMGADn+QqudbHsc5Hnd4…' │ 'v1.GGF0lyrefG6Mut4QH2y0Py7Zl…' │
│ 1       │ 'INV-20981' │ 'v1.2BhB/3EtqlRdw+arKW6Y8ZC4V…' │ 'v1.k7qFh/qhru8b/N1teCCuEnABB…' │
└─────────┴─────────────┴─────────────────────────────────┴─────────────────────────────────┘

And one supplier's invoices are found through the blind index, in any capitalisation, without decrypting a single row to search:

node show.mjs --supplier ACCOUNTS@lumen-supplies.example

python show.py --supplier ACCOUNTS@lumen-supplies.example

// Read the invoices back. Without DATA_KEY, the database shows only ciphertext.
//   node show.mjs                       every invoice, decrypted
//   node show.mjs --raw                 what's actually stored
//   node show.mjs --supplier a@b.com    one supplier's invoices, found by blind index (nothing decrypted to search)
import { openDatabase } from "./database.mjs";
import { blindIndex, decrypt } from "./encryption.mjs";

const db = openDatabase();
const args = process.argv.slice(2);
const supplier = args.includes("--supplier") ? args[args.indexOf("--supplier") + 1] : null;
const rows = supplier
  ? db.prepare("SELECT * FROM invoices WHERE supplier_email_idx = ? ORDER BY received_at").all(blindIndex(supplier))
  : db.prepare("SELECT * FROM invoices ORDER BY received_at").all();

if (args.includes("--raw")) console.table(rows.map((r) => ({ invoice: r.invoice_number, supplier_email_enc: `${r.supplier_email_enc?.slice(0, 28)}…`, body_enc: `${r.body_enc?.slice(0, 28)}…` })));
else console.table(rows.map((r) => ({
  invoice: r.invoice_number,
  amount: r.amount_due,
  due: r.due_date,
  supplier: decrypt(r.supplier_name_enc, "supplier_name") ?? decrypt(r.supplier_email_enc, "supplier_email"),
  email: decrypt(r.supplier_email_enc, "supplier_email"),
})));

"""Read the invoices back.  python show.py  |  python show.py --raw  |  python show.py --supplier a@b.com"""
import sys

from database import open_database
from encryption import blind_index, decrypt

db = open_database()
args = sys.argv[1:]
if "--supplier" in args:
    rows = db.execute("SELECT * FROM invoices WHERE supplier_email_idx = ? ORDER BY received_at", (blind_index(args[args.index("--supplier") + 1]),)).fetchall()
else:
    rows = db.execute("SELECT * FROM invoices ORDER BY received_at").fetchall()

for r in rows:
    if "--raw" in args:
        print(f"{r['invoice_number']}  {r['supplier_email_enc'][:32]}…  {r['body_enc'][:32]}…")
    else:
        name = decrypt(r["supplier_name_enc"], "supplier_name")
        email = decrypt(r["supplier_email_enc"], "supplier_email")
        print(f"{r['invoice_number']}  {r['amount_due']}  due {r['due_date']}  {name or email} <{email}>")

With the wrong key, decryption fails with unable to authenticate data instead of returning nonsense.

7. Go live

Deploy the server behind HTTPS and put its address in Dashboard → Inbound webhook (for example https://api.yourcompany.com/webhooks/email).
Keep DATA_KEY out of the code and the database: in your host's secret store (AWS Secrets Manager, Google Secret Manager, Doppler, a Kubernetes secret). Back it up separately from the database.
Use your own database if you like: PostgreSQL or MySQL work the same way. The encrypted columns are plain TEXT.
Rotating the key: encrypt new rows as v2. with the new key, decrypt by prefix, and re-encrypt old rows in the background.
Send a real invoice to invoices@ and watch it appear in Dashboard → Deliveries and in your table.

Questions

Why not just turn on disk encryption for the database?

Do that too. But disk encryption only protects a stolen disk: anyone who can run a query, read a backup or see a log of the data sees everything. Field-level encryption keeps the sensitive columns unreadable to all of them, and only the code holding the key can read them.

Can I search encrypted fields?

For exact matches, yes: store a blind index (a keyed hash) beside the ciphertext, as this tutorial does for the supplier's email. Partial-text search over encrypted data isn't possible this way; keep fields you need to search that way unencrypted, or search them after decrypting.

What happens if I lose the key?

The encrypted columns can't be recovered, by you or anyone else. That's the point, so back the key up somewhere other than the database.

Does GhostParse keep a copy of the email?

No. Email is parsed in memory and delivered to your webhook; we keep a short log of metadata such as the time and the result, never the message. Once it's in your database, yours is the only copy.

Can I do this without writing a server?

If you don't need encryption, yes: send the parsed fields straight to Google Sheets, Make or n8n with a routing rule. For encrypted storage you need code that holds the key, like this server.