How We Scaled ShowForm to 5M+ Form Submissions on Shopify Without Database Degradation

When building public Shopify SaaS applications installed across thousands of high-traffic merchants, traffic spikes during Black Friday Cyber Monday (BFCM) can generate thousands of concurrent form submissions per minute. By decoupling edge ingestion from asynchronous worker queues, ShowForm scaled past 5,000,000 processed submissions with zero data loss and an interaction-to-next-paint (INP) latency under 16ms.

ShowForm Scaling Architecture Comparison

| Metric | Synchronous SQL Writes | Decoupled SQS + Redis Worker Queue |

| :--- | :--- | :--- |

| Peak Throughput | 180 submissions/sec (Connection limits) | 4,500+ submissions/sec |

| Client Response Time | 380ms (Waits for DB roundtrip) | 28ms (Edge ACK to client) |

| Database Load (CPU) | 92% during flash sales | Maintained steady below 22% |

| Failure Rate | 2.4% during DB connection exhaustion | 0.00% (Durable dead-letter queue) |

1. High-Throughput Edge Ingestion Pipeline

To ensure the storefront popup never blocks buyer checkout, the browser client sends form responses directly to a lightweight Cloudflare Worker or AWS Lambda function that immediately enqueues the payload into AWS SQS:

import { SQSClient, SendMessageCommand } from "@aws-sdk/client-sqs";

const sqs = new SQSClient({ region: "us-east-1" });

export async function handleFormSubmission(event: APIGatewayEvent) {
  const payload = JSON.parse(event.body);

  // Validate Shopify store origin and token
  if (!isValidShopifyOrigin(payload.shopDomain)) {
    return { statusCode: 403, body: "Forbidden" };
  }

  // Push to high-throughput FIFO queue
  await sqs.send(new SendMessageCommand({
    QueueUrl: process.env.SUBMISSION_QUEUE_URL,
    MessageBody: JSON.stringify(payload),
    MessageGroupId: payload.shopDomain,
    MessageDeduplicationId: payload.submissionId,
  }));

  // Return immediate 202 Accepted to browser
  return {
    statusCode: 202,
    body: JSON.stringify({ status: "queued", submissionId: payload.submissionId }),
  };
}

2. Worker Queue Processing & Webhook Delivery

Background workers batch-process incoming messages, write records to PostgreSQL in transactional chunks of 500, and dispatch outbound webhooks to merchant CRMs (Klaviyo, Mailchimp, Omnisend):

export async function processBatchSubmissions(messages: SQSMessage[]) {
  const batchValues = messages.map(m => JSON.parse(m.Body));

  // Single bulk INSERT statement prevents connection pool exhaustion
  await prisma.formSubmission.createMany({
    data: batchValues,
    skipDuplicates: true,
  });

  // Dispatch asynchronous webhooks via BullMQ
  for (const item of batchValues) {
    await webhookQueue.add("dispatch_crm", item, { attempts: 5, backoff: { type: "exponential", delay: 2000 } });
  }
}

3. Engineering Partnerships and Ecosystem Impact

The high-volume reliability of ShowForm powers Shopify Plus merchants engineered by our sister studio StoresForge, verifying QByteSoft's end-to-end capability in architecting resilient e-commerce infrastructure.

Frequently Asked Questions

How does ShowForm achieve zero impact on Shopify Core Web Vitals?

The ShowForm script tag uses asynchronous deferral and requestIdleCallback wrappers, keeping total script execution time below 30ms and preventing layout shifts (zero CLS).

What happens if an external ESP (like Klaviyo) experiences downtime?

Outbound notifications use exponential backoff retry queues in BullMQ with dead-letter storage, ensuring no customer lead is ever dropped.