Telehealth Platform Development: Architecture for Video Visits, Scheduling, and Clinical Documentation

Telehealth grew from a niche offering to a mainstream care delivery channel between 2020 and 2022, and the market has since bifurcated sharply: platforms built hastily during the pandemic, showing their limitations, and platforms built with clinical workflow in mind, becoming the default channel for certain care types. Building in the second category requires understanding what telehealth platforms actually need to do beyond the video call.

This post covers the architecture of a production telehealth platform — the technical components, the HIPAA considerations at each layer, and the clinical workflow decisions that determine whether the platform actually improves patient care.

The clinical workflow model

A telehealth visit has a different lifecycle than an in-person visit, and the platform architecture must reflect that lifecycle. The stages:

1. Scheduling: Patient books an appointment — self-service or via staff. Availability is checked against provider schedules. Insurance eligibility may be verified before or after booking.
2. Pre-visit intake: Patient receives intake forms (demographics update, chief complaint, medication list, consent to telehealth). Completed intake is available to the provider before the visit begins.
3. Pre-visit preparation: Provider reviews intake, previous visit notes, and relevant history. The platform surfaces relevant information without requiring the provider to navigate through the EHR.
4. The video visit: Provider and patient connect. The session is recorded if the patient consents (and state law permits). Provider may share screen (for showing imaging, test results, patient education materials).
5. Clinical documentation: During or immediately after the visit, the provider completes the visit note, orders any tests or prescriptions, and sets the follow-up plan.
6. Post-visit: Visit summary sent to patient. Referrals routed. Orders transmitted to lab, pharmacy, or specialist. Documentation synced back to the EHR.

Platforms that focus exclusively on the video call and treat everything else as an afterthought end up with providers doing workarounds — completing notes in a separate EHR, managing their own follow-up, and losing the efficiency gains that telehealth should deliver.

Video infrastructure

Build vs buy for video

Building a WebRTC-based video stack from scratch is possible, but it is an enormous undertaking. For most telehealth platforms, the right decision is to use a HIPAA-compliant video SDK or platform — Twilio Video, Daily.co, or Amazon Chime SDK — and build the clinical workflow around it.

The HIPAA requirement: the video provider must sign a BAA. Twilio Video, Daily.co, Amazon Chime, and Zoom for Healthcare all offer BAAs for healthcare use cases. Zoom's standard consumer product does not have a BAA and should not be used for telehealth.

Video session architecture with Twilio Video

// Generate access token for a video session
async function createVideoSessionToken(
  participantId: string,
  roomId: string,
  role: 'patient' | 'provider'
): Promise {
  const identity = \`\${role}:\${participantId}\`

  const token = new AccessToken(
    TWILIO_ACCOUNT_SID,
    TWILIO_API_KEY_SID,
    TWILIO_API_KEY_SECRET,
    { identity, ttl: 3600 }  // 1 hour max session length
  )

  const videoGrant = new VideoGrant({
    room: roomId,
  })

  token.addGrant(videoGrant)
  return token.toJwt()
}

// Create the room with HIPAA-appropriate settings
async function createVideoRoom(appointmentId: string): Promise {
  const room = await twilioClient.video.rooms.create({
    uniqueName: \`appt-\${appointmentId}\`,
    type: 'go',               // peer-to-peer for 2-participant visits, lower latency
    maxParticipants: 5,       // allow up to 5 (provider, patient, interpreter, chaperone, supervisor)
    recordParticipantsOnConnect: false,  // explicit consent before enabling recording
    statusCallback: \`\${BASE_URL}/api/video/status\`,
    statusCallbackMethod: 'POST',
  })
  return room.uniqueName
}

Waiting room implementation

The waiting room is the patient's experience before the provider joins. It should:

• Confirm the patient's device (camera, microphone, network) works before the visit
• Show estimated wait time if the provider is running behind
• Allow the patient to complete any outstanding intake forms
• Notify the provider when the patient has arrived and is ready

// Patient joins the waiting room — provider is notified
async function patientJoinedWaitingRoom(appointmentId: string, patientId: string): Promise {
  await db.appointments.update({
    where: { id: appointmentId },
    data: { patientArrivedAt: new Date() },
  })

  const appointment = await db.appointments.findUnique({
    where: { id: appointmentId },
    include: { provider: { select: { userId: true } } },
  })

  // Push notification to provider
  await pushNotificationService.send({
    userId: appointment!.provider.userId,
    title: 'Patient Ready',
    body: \`Your \${format(appointment!.startTime, 'h:mm a')} patient is ready\`,
    data: { appointmentId, type: 'patient-arrived' },
  })

  // In-app WebSocket notification for the provider dashboard
  await wsServer.sendToUser(appointment!.provider.userId, {
    type: 'patient-arrived',
    appointmentId,
    arrivedAt: new Date().toISOString(),
  })
}

Pre-visit intake

The intake form is the first clinical touchpoint and the data it collects directly affects care quality. The intake system must handle:

• Structured data fields (medications, allergies, symptoms with severity and duration)
• Branching logic — answers to one question determine which questions appear next (positive COVID exposure → quarantine status questions)
• Photo uploads for skin conditions, wound images, medication bottles
• Consent forms with e-signature
• Data pre-population from previous visits for returning patients

interface IntakeForm {
  formId: string
  appointmentId: string
  version: number
  sections: IntakeSection[]
  signature?: {
    signerName: string
    relationship: 'self' | 'guardian' | 'caregiver'
    signedAt: Date
    consentText: string
    signatureDataUrl: string  // base64 PNG of the drawn signature
  }
}

interface IntakeSection {
  sectionId: string
  title: string
  fields: IntakeField[]
  displayCondition?: {   // show this section only if condition is met
    fieldId: string
    value: string | boolean
  }
}

// Intake form data is encrypted at rest (field-level for PHI fields)
async function saveIntakeResponse(response: IntakeForm): Promise {
  const encrypted = await encryptPHIFields(response, PHI_FIELD_IDS)
  await db.intakeResponses.upsert({
    where: { appointmentId: response.appointmentId },
    create: { ...encrypted, submittedAt: new Date() },
    update: { ...encrypted, updatedAt: new Date() },
  })
}

Clinical documentation

The visit note is a SOAP note (Subjective, Objective, Assessment, Plan) or equivalent clinical documentation structure. The documentation system must support:

• Structured and free-text fields — structured for coded diagnoses (ICD-10), medications, and orders; free text for the narrative portions
• Auto-population from intake data — the patient's chief complaint, medication list, and allergies pre-populate the relevant SOAP sections
• Clinical decision support — prompts for preventive care, drug-drug interactions, diagnostic considerations
• Voice-to-text note entry — providers who dictate notes rather than type them
• Note templates — specialty-specific and visit-type-specific templates that structure the documentation

AI-assisted documentation

This is an area where AI genuinely improves clinical workflow. An AI scribe that listens to the video visit (with patient consent) and generates a draft SOAP note for the provider to review reduces documentation time significantly. The implementation:

// Start recording and transcription for AI documentation
async function startAIDocumentation(sessionId: string, appointmentId: string): Promise {
  // Transcription via Deepgram with medical model (better medical vocabulary accuracy)
  const transcriptionJob = await deepgram.transcription.preRecorded.transcribeUrl({
    url: videoSession.recordingUrl,
    options: {
      model: 'nova-medical',
      smart_format: true,
      speaker_labels: true,   // distinguish provider vs patient speech
      utterances: true,
    }
  })

  // After transcription, generate SOAP note draft via LLM
  const transcript = transcriptionJob.results?.utterances?.map(u =>
    \`[\${u.speaker === 0 ? 'Provider' : 'Patient'}]: \${u.transcript}\`
  ).join('\n')

  const soapDraft = await generateSOAPNote(transcript, await getPatientContext(appointmentId))

  await db.visitNotes.create({
    data: {
      appointmentId,
      draftContent: soapDraft,
      transcriptContent: transcript,
      generatedAt: new Date(),
      status: 'draft',   // must be reviewed and signed by provider
    }
  })
}

The critical point: AI-generated notes are always drafts that a licensed provider must review, edit, and sign. The note cannot be final until a provider has accepted clinical responsibility for it. The system must enforce this — there should be no pathway for an AI-generated note to enter the EHR as a finalized document without provider review.

EHR integration

The telehealth platform needs to read from and write to the EHR. For most modern EHRs, this is the FHIR R4 API. The integration reads:

• Patient demographics and insurance (to pre-populate intake)
• Problem list, medication list, allergy list (to pre-populate documentation)
• Previous visit notes (to give provider context)
• Imaging and lab results (to share during the visit)

The integration writes:

• Visit note (as a DocumentReference FHIR resource)
• Orders (Prescription → MedicationRequest, Lab order → ServiceRequest)
• Diagnosis codes (Condition resources)

// Write completed visit note to EHR via FHIR
async function syncVisitNoteToEHR(note: CompletedVisitNote): Promise {
  const document: fhir4.DocumentReference = {
    resourceType: 'DocumentReference',
    status: 'current',
    type: {
      coding: [{
        system: 'http://loinc.org',
        code: '34137-0',   // LOINC code for outpatient note
        display: 'Outpatient Note'
      }]
    },
    subject: { reference: \`Patient/\${note.ehrPatientId}\` },
    date: note.signedAt.toISOString(),
    author: [{ reference: \`Practitioner/\${note.provider.ehrProviderId}\` }],
    content: [{
      attachment: {
        contentType: 'text/plain',
        data: Buffer.from(note.content).toString('base64'),
      }
    }]
  }

  await ehrClient.create(document)
}

Scheduling system

The scheduling system must respect provider availability rules that go beyond simple open slots:

• Buffer time between appointments for documentation
• Visit type restrictions — provider can only schedule telehealth visits for established patients, for example
• State licensure — provider can only see patients in states where they are licensed, enforced at booking time based on the patient's state of residence
• Insurance panel restrictions — provider may not accept all insurance plans
• Appointment type duration mapping — new patient visits are 45 minutes, follow-ups are 20 minutes

The state licensure check is particularly important for telehealth. Unlike in-person care, patients may attempt to book with a provider in another state where the provider is not licensed to practice. This must be checked at booking time, not discovered later.

Telehealth platforms that handle all of these requirements — not just the video call — are the ones that clinical teams actually adopt. If you are building or evaluating a telehealth platform and want a technical review of the architecture, we have built clinical-grade telehealth infrastructure and can help you evaluate what you have and what needs improvement.