Key Takeaways

• Real-time kinematic body tracking has matured to the point where a smartphone camera can estimate dozens of 3D joint landmarks at video frame rates, turning any living room into a movement lab. For physical therapy, that unlocks remote exercise guidance, objective range-of-motion measurement, and continuous adherence monitoring at a fraction of the cost of clinic-based motion capture.
• Whether your app needs FDA clearance is the single most consequential decision you will make, and it hinges almost entirely on intended use. The same body-tracking engine can power an unregulated general wellness product or a regulated Software as a Medical Device, and the January 2026 FDA guidance updates have reshaped exactly where that line falls.
• The technology is powerful but imperfect. Markerless pose estimation approaches lab-grade accuracy for movement patterns and relative range of motion, yet still carries meaningful error at extreme joint angles, so your clinical claims, your validation strategy, and your regulatory posture all have to be calibrated to what the technology can actually defend.

The Living Room Becomes a Movement Lab

For most of the history of physical therapy, measuring how a patient moves required either a trained clinician with a goniometer or a room full of infrared cameras and reflective markers costing six figures. Both approaches share the same limitation: they only work when the patient is physically present in a clinical setting. The moment the patient goes home, where the overwhelming majority of prescribed rehabilitation actually happens, the clinician loses all visibility into whether exercises are being done, done correctly, or done at all.

That blind spot is expensive. Musculoskeletal conditions are the leading cause of disability worldwide, with the World Health Organization estimating that roughly 1.71 billion people are affected by musculoskeletal disorders at some point. In the United States alone, chronic pain, much of it musculoskeletal, costs the healthcare system an estimated $635 billion a year. And the recurring problem across all of it is adherence. Patients forget exercises, perform them incorrectly, lose motivation without feedback, and quietly drop out of their home programs, which leads to slower recovery, repeat injuries, and avoidable surgeries.

Real-time kinematic body tracking is what makes the home a viable place to solve that problem. Modern pose-estimation models can take an ordinary RGB video stream from a phone or tablet and infer the positions of dozens of body landmarks in three dimensions, many times per second, without a single marker on the patient’s body. From those landmarks an app can compute joint angles, count repetitions, evaluate movement quality, and give the patient immediate corrective feedback, all on a device they already own. 

The investment community has noticed. The digital musculoskeletal care market reached approximately $4.1 to $4.4 billion in 2024 and is growing at double-digit annual rates, with companies like Hinge Health, Sword Health, Kaia Health, and IncludeHealth building computer-vision-powered platforms. Hinge Health went so far as to acquire a computer vision company to strengthen its movement-tracking capabilities.

For businesses, the opportunity is clear, but so is the complexity. Building a physical therapy app on body-tracking APIs sits at the intersection of two demanding disciplines: the computer vision and biomechanics that make the tracking accurate, and the FDA regulatory framework that determines whether you can legally make the claims you want to make. In this post, we will walk through the technology, the regulatory decision that defines your entire project, the engineering required to build the app responsibly, and a roadmap for getting it to market.

Understanding Real-Time Kinematic Body Tracking

Before tackling regulation, it helps to understand what these APIs actually do and where their limits lie, because those limits directly shape what you can claim and how you must validate.

Kinematic body tracking, often called markerless motion capture or human pose estimation, uses machine learning models trained to locate anatomical keypoints in an image. Give the model a frame of video and it returns a set of landmark coordinates: shoulders, elbows, wrists, hips, knees, ankles, and more. String those frames together and you have a real-time skeletal model of a moving person. From the geometry of those landmarks, software computes the kinematics that matter clinically, including joint angles, range of motion, movement velocity, symmetry between limbs, and the quality of a movement pattern compared to a reference.

Several mature APIs put this capability within reach of any development team. On the Google and Android side, the ML Kit Pose Detection API tracks 33 body landmarks in real time, built on the same BlazePose research that powers MediaPipe Pose, which infers 33 three-dimensional landmarks from RGB frames and runs on most modern mobile phones. 

ML Kit even offers developers a choice between a faster model and a more accurate model, letting you trade latency against precision depending on the clinical task. On the Apple side, the Vision framework provides body pose detection with a confidence value for each landmark, and can run on still images or live camera feeds, while ARKit offers live motion capture designed specifically for real-time applications using the rear camera. These platform-native options can be supplemented or replaced with open-source pose estimators and commercial computer-vision SDKs depending on the accuracy, platform, and licensing requirements of the product.

Here is where honesty about the technology becomes essential, because overclaiming is both a clinical risk and a regulatory one. The peer-reviewed literature shows that markerless pose estimation has become genuinely useful for rehabilitation, but it is not yet equivalent to laboratory motion capture in every respect. 

Validation studies have found that markerless systems can produce joint-angle measurements within roughly five degrees of a gold-standard marker-based system for tasks like gait analysis, and can reproduce the shape of a movement with high reliability. A recent gait study reported excellent test-retest reliability and root-mean-square errors under three degrees for hip and knee angles. 

At the same time, other research is candid about the gaps: one systematic assessment of eleven open-source pose estimators found mean per-joint position errors ranging from 72 to 122 millimeters in two dimensions and 146 to 249 millimeters in three dimensions, and a broader review concluded that while temporal and spatial measures are often equivalent to marker-based capture, joint center locations and absolute joint angles are not yet sufficiently accurate for every clinical application.

The practical reading of this evidence is nuanced rather than discouraging. Markerless tracking is well suited to recognizing movement patterns, scaling relative range of motion, counting repetitions, and assessing whether a movement is improving over time. It is less reliable for precise absolute angle measurement at the extremes of joint excursion. 

A well-designed physical therapy app builds on the technology’s strengths and designs around its weaknesses, and just as importantly, it makes only the clinical claims the underlying accuracy can defend. That last point is not merely good engineering. As we will see, it is the hinge on which your entire regulatory strategy turns.

The Decision That Defines Your Entire Project: Wellness or Medical Device

Every other decision in this project flows from one regulatory question: is your app a general wellness product, which the FDA does not regulate as a medical device, or is it Software as a Medical Device, which it does? Getting this classification right at the outset determines your timeline, your budget, your documentation burden, and the claims you can legally make. Getting it wrong is one of the most expensive mistakes a digital health company can make.

The FDA defines Software as a Medical Device as software intended for one or more medical purposes that performs those purposes without being part of a hardware device. The key phrase is “intended for a medical purpose.” A general wellness product, by contrast, is one intended only to maintain or encourage a healthy lifestyle and unrelated to diagnosing, treating, curing, mitigating, or preventing a disease or condition. The exact same body-tracking engine can sit inside either kind of product. What changes is the intended use, judged objectively from your labeling, your marketing, and the claims you make.

Consider two physical therapy apps built on identical pose-detection technology. The first guides users through general flexibility and strengthening routines, tracks their range of motion over time as a fitness metric, and never references any disease or injury. That app is likely a general wellness product outside the scope of FDA device regulation. 

The second is intended to treat patients recovering from a specific knee surgery, measures their progress against clinical thresholds, and adjusts their therapy based on those measurements. That app is making a medical claim, is intended to treat a condition, and is very likely a regulated medical device. The technology is the same. The intended use is worlds apart.

This distinction became both clearer and more favorable to developers in early 2026. On January 6, 2026, the FDA issued updated guidance on both General Wellness: Policy for Low Risk Devices and Clinical Decision Support Software, and the day after withdrew its older guidance on SaMD clinical evaluation. The updated general wellness guidance preserves the FDA’s longstanding policy of not regulating low-risk general wellness products as medical devices, while clarifying the boundaries for modern wearables and software. 

The thrust of the 2026 updates leans toward a more deregulatory posture, with the FDA exercising enforcement discretion over a broader set of low-risk products and exempting certain non-invasive monitoring functions, provided they avoid clinical claims, disease references, and diagnostic thresholds. FDA leadership framed the changes as an effort to make it easier to bring certain wearables and decision-support software to market without strict premarket review.

That deregulatory drift is good news for many physical therapy app builders, but it comes with a crucial caveat that is easy to miss. The line is defined by intended use and risk, not by technology, and it is genuinely possible to cross it without realizing it. The moment your marketing references a specific condition, the moment your app outputs a measurement tied to a clinical threshold, the moment it makes a treatment recommendation that a patient would rely on without a clinician’s independent review, you risk moving from the unregulated wellness category into regulated medical device territory. 

Legal commentators analyzing the 2026 guidance have warned that significant areas of ambiguity remain about exactly where that line sits. This is precisely why classification should be settled deliberately, with experienced regulatory input, before a single feature is built.

When Your PT App Is a Medical Device: The Regulatory Pathways

Suppose your intended use does place your app in the medical device category. That is not a dead end; it is a well-trodden path that thousands of digital health products have walked, and the right healthcare app development partner can map it with you. But it does mean committing to a regulatory pathway and the engineering discipline that comes with it.

Most software medical devices reach the United States market through one of two FDA pathways. The first and most common is 510(k) premarket notification, which applies when your device is substantially equivalent to a legally marketed predicate device. If another cleared product already does something similar to what your app does, you demonstrate that your app is as safe and effective as that predicate, and you can often avoid the far more demanding premarket approval process required for high-risk devices. 

As of late 2022, all 510(k) submissions must be filed electronically through the FDA’s CDRH Portal using the standardized eSTAR format, which has made the process more predictable. The second pathway is De Novo classification, which exists for genuinely novel low-to-moderate-risk devices that have no suitable predicate. Because real-time kinematic PT apps are a relatively new category, some products in this space may find De Novo to be the appropriate route, establishing a new device classification that later products can then use as their own predicate.

Whichever pathway applies, building a regulated SaMD imposes a set of engineering and documentation obligations that general consumer apps never face. These products must adhere to recognized software lifecycle and risk standards, including IEC 62304 for software lifecycle processes and ISO 14971 for risk management, and operate under a quality management system aligned with ISO 13485.

 The submission itself typically requires a clear device description and intended-use statement, a substantial-equivalence comparison to your predicate, software verification and validation data, evidence that the device performs as claimed, labeling with appropriate instructions and warnings, and a cybersecurity risk management plan. The FDA’s updated cybersecurity guidance has made the last of these a make-or-break element rather than an afterthought.

The throughline that experienced developers understand is that none of this can be bolted on at the end. The most effective approach to building a regulated PT app integrates regulatory requirements from the first sprint, so that quality, documentation, and validation are produced continuously throughout development rather than reconstructed under deadline pressure before submission. 

This is the same discipline that defines serious digital therapeutics app development, where evidence and compliance are designed into the product from the beginning. Retrofitting compliance into an app that was architected without it is slow, expensive, and risky, and it is the single most common way that promising digital health products stall on their way to market.

Designing the App: From Landmarks to Clinical Value

With the regulatory frame established, here is what it actually takes to turn a stream of pose landmarks into a physical therapy product that clinicians trust and patients use. Each layer below builds on the one before it.

The Tracking and Kinematics Layer

At the foundation sits the body-tracking engine and the math that turns its raw output into clinical metrics. Selecting the right pose-estimation API is the first decision, and it should be driven by the clinical task. An app assessing gross movement patterns and counting repetitions can tolerate a faster, lighter model; an app measuring range of motion for progress tracking needs the most accurate model available and careful attention to camera positioning. 

On top of the raw landmarks, you build the kinematics engine that computes joint angles from landmark geometry, smooths the inevitable jitter in frame-to-frame detection, and handles the practical realities of home use: poor lighting, partial occlusion when one body part hides another, varying camera angles, and patients who position their device inconsistently. A robust approach often leans on angular relationships between joints rather than absolute positions, since relative geometry is more resilient to differences in scale, viewpoint, and skeletal jitter than raw coordinates.

The Clinical Logic Layer

Above the kinematics sits the clinical intelligence that makes the app therapeutically useful. This is where prescribed exercises are defined, where correct form is encoded as acceptable ranges of joint movement, where repetitions are counted and quality is scored, and where the app decides what feedback to give. Crucially, this is also the layer most likely to carry regulatory weight. 

Feedback that simply helps a user perform a general exercise looks like wellness. Logic that evaluates a patient’s movement against clinical criteria tied to a diagnosed condition, and adjusts a treatment plan accordingly, looks like a medical device. The design of this layer should be made hand in hand with your regulatory strategy, not independently of it, because the same feature can land on either side of the FDA line depending on how it is framed and what it claims.

The Real-Time Feedback Layer

The feature that distinguishes a kinematic PT app from a video of exercises is immediate, actionable feedback, and it is what drives the adherence gains that make these products valuable. As the patient moves, the app should tell them in real time whether they are hitting the target range, whether their form is breaking down, and how to correct it, using visual overlays, audio cues, or both. 

Designing this well is a genuine human-factors challenge. Feedback must be timely enough to be useful mid-movement, clear enough for a patient with no clinical training to act on, and encouraging rather than discouraging, since motivation is often the real determinant of whether a home program succeeds. For regulated products, this feedback layer may also be subject to usability testing, which the FDA increasingly expects for software whose outputs patients act on directly.

The Clinician and Data Layer

Finally, a serious physical therapy app connects the patient’s home activity back to their care team. Therapists need a dashboard showing which patients are doing their exercises, how their range of motion is trending, where form problems are recurring, and which patients may need a check-in. This remote-monitoring capability is often the core of the product’s value to providers and payers, and it benefits from integration with clinical systems through standards like FHIR so that movement data flows into the broader record rather than living in a silo. This layer also carries the full weight of health-data compliance, which deserves its own discussion.

Security, Privacy, and Trust Are Foundational

Everything a physical therapy app touches, from video of a patient exercising in their home to measurements of their physical capabilities, is sensitive health information, and protecting it is a foundational design constraint rather than a feature added at the end. For any product handling protected health information in the United States, HIPAA compliance has to be engineered in from the first line of code, which in practice means end-to-end encryption of data in transit and at rest, strong authentication, role-based access controls that enforce minimum-necessary access, and comprehensive audit logging.

Video data raises the stakes further. A camera-based app is, by definition, capturing imagery of people in their homes, sometimes in minimal clothing during exercises. That demands particular care: being deliberate about whether raw video is ever stored or transmitted versus processed on-device into abstract landmark data, obtaining clear and specific consent, and minimizing the data you collect to only what the clinical purpose genuinely requires. On-device processing, where the pose estimation runs locally and only derived metrics leave the device, is often both a privacy advantage and a performance one, since it avoids transmitting sensitive video at all.

It is also worth designing with the wider regulatory landscape in mind. State consumer health-privacy laws increasingly reach health data that falls outside HIPAA, and a camera-based PT app that operates in the consumer wellness space may find itself subject to those laws even when HIPAA does not apply. Building to a high standard of data privacy and ethical handling of user information is not only the compliant choice but the one that earns the trust these products depend on. Patients will only do their exercises in front of a camera if they believe the company on the other end will handle that intimate data responsibly.

A Practical Roadmap to Market

Pulling the technology, the regulation, and the engineering together, here is a sequence that takes a kinematic PT app from concept to launch while managing risk at each stage.

Step 1: Define Intended Use and Settle Classification

Before anything else, decide precisely what your app is for and who it serves, and use that to settle the wellness-versus-device classification with experienced regulatory input. This single decision shapes every downstream choice, from the claims your marketing can make to the documentation your engineers must produce. Resist the temptation to defer it; the cost of discovering late that you built a medical device while believing you were building a wellness app is measured in months and millions.

Step 2: Validate the Technology Against Your Claims

Establish early what your chosen body-tracking approach can and cannot measure accurately for your specific use case, and align your claims to that reality. If you intend to measure range of motion as a clinical endpoint, you need validation data showing your system’s accuracy for the relevant joints and movements. If your claims are about movement patterns and adherence rather than precise angles, your validation can be scoped accordingly. Either way, the validation should precede the marketing, not follow it.

Step 3: Build the Core Loop With Compliance Engineered In

Develop the foundational tracking, kinematics, feedback, and data layers as an integrated whole, with HIPAA safeguards and, if you are building a regulated device, your quality system and documentation practices in place from the first sprint. Get the core patient experience working and trusted before expanding scope. A reliable app that accurately tracks a focused set of exercises and gives genuinely useful feedback is worth far more than a sprawling, feature-rich app that patients do not believe.

Step 4: Pursue the Regulatory Pathway in Parallel

If your app is a medical device, run your 510(k) or De Novo preparation alongside development rather than after it, assembling your device description, validation data, software documentation, and cybersecurity plan as the product takes shape. For regulated products, early engagement with the FDA can clarify expectations and reduce the risk of costly surprises late in the process. Remember too that medical device apps face a second layer of review from the Apple and Google app stores, which impose their own scrutiny on health software, so plan for that dual approval rather than being surprised by it.

Step 5: Launch, Monitor, and Improve

Compliance and quality do not end at launch. Regulated devices require post-market surveillance, and all health apps benefit from monitoring real-world performance, gathering outcome data, and improving the product over time. If your app uses machine learning models that you intend to update, the FDA’s frameworks for predetermined change control let you plan for those updates in a way that maintains safety oversight without requiring a fresh submission for every improvement.

The Opportunity Is Real, and So Is the Discipline It Demands

Real-time kinematic body tracking has genuinely changed what a physical therapy app can do. A patient can now stand in front of a phone, perform their prescribed exercises, receive immediate corrective feedback, and have their progress flow back to their therapist, all without leaving home. Against a backdrop of 1.71 billion people with musculoskeletal conditions and a digital MSK market already measured in billions and growing fast, the commercial and clinical opportunity is substantial and well established.

But the same qualities that make this category valuable, its proximity to real clinical decisions and real patient outcomes, are what make it demanding to build responsibly. The wellness-versus-device classification will define your entire project, and the 2026 regulatory shifts have moved that line in ways every builder needs to understand precisely. The technology is powerful but imperfect, which means your claims and your validation have to be honest about what markerless tracking can actually defend. And the whole thing has to be engineered with security, quality, and compliance designed in from the start, because none of it can be retrofitted afterward.

That is the hard part, and it is the part worth getting right: the judgment to classify the product correctly, the rigor to validate the technology against defensible claims, and the engineering discipline to build a compliant, secure, clinically credible app that physical therapists and patients will actually trust. If you are building a physical therapy app on real-time body tracking and want a partner who understands both the computer vision and the regulatory pathway, the team at Dogtown Media builds compliant, FDA-ready health applications from the ground up.

Frequently Asked Questions

Does a physical therapy app that uses body tracking need FDA clearance?

It depends entirely on intended use, not on the technology. If the app is intended only to support general fitness, flexibility, or wellness and makes no claims about diagnosing, treating, or preventing a specific condition, it is likely a general wellness product outside FDA device regulation. If it is intended to treat or manage a specific medical condition, measures progress against clinical thresholds, or makes treatment recommendations a patient would rely on, it likely qualifies as Software as a Medical Device and requires FDA clearance. The January 2026 FDA guidance updates clarified these boundaries and leaned somewhat more permissive for low-risk wellness products, but the line is still defined by intended use and risk, and it is possible to cross it inadvertently through marketing claims or clinical features.

How accurate is markerless body tracking compared to a real motion capture lab?

It is good and getting better, but not yet equivalent in every respect. Validation studies have found markerless systems can measure some joint angles within about five degrees of gold-standard marker-based systems and can reliably capture movement patterns and relative range of motion. However, research also shows meaningful error in absolute joint positions and at extreme joint angles, with some studies reporting per-joint position errors of several centimeters in three dimensions. The practical implication is that markerless tracking is well suited to recognizing movement patterns, counting repetitions, and tracking relative progress, while precise absolute angle measurement at the limits of motion remains a weaker area. Your clinical claims should match what the technology can defend.

What body-tracking APIs are available for building a PT app?

Several mature options exist. Google’s ML Kit Pose Detection API and the underlying MediaPipe Pose solution track 33 three-dimensional body landmarks in real time on mobile devices, with options to trade speed against accuracy. Apple’s Vision framework provides body pose detection with per-landmark confidence values and works on both still images and live video, while ARKit offers live motion capture optimized for real-time use with the rear camera. These platform-native tools can be supplemented with open-source pose estimators or commercial computer-vision SDKs depending on your accuracy, platform, and licensing needs.

What is the difference between the 510(k) and De Novo pathways?

The 510(k) pathway applies when your device is substantially equivalent to an existing legally marketed device, called a predicate. You demonstrate that your app is as safe and effective as that predicate, which is generally faster than the most demanding approval routes. The De Novo pathway exists for novel low-to-moderate-risk devices that have no suitable predicate to compare against. Because real-time kinematic PT apps are a relatively new category, some products may not find a clean predicate and may pursue De Novo classification, which then establishes a new device type that later products can use as their predicate.

What standards and documentation does a regulated PT app require?

Regulated Software as a Medical Device must generally follow recognized software and risk standards, including IEC 62304 for the software lifecycle and ISO 14971 for risk management, under a quality management system aligned with ISO 13485. A typical submission includes a device description and intended-use statement, a substantial-equivalence comparison to a predicate where applicable, software verification and validation data, performance evidence supporting your claims, labeling with instructions and warnings, and a cybersecurity risk management plan. The most important principle is to produce this documentation continuously from the first sprint rather than assembling it at the end.

Is patient video data a privacy risk, and how should it be handled?

Yes, camera-based apps capture sensitive imagery of patients in their homes, so privacy must be a foundational design decision. Best practices include processing video into abstract landmark data on the device whenever possible so that raw video never leaves the phone, obtaining clear and specific consent, minimizing the data collected to only what the clinical purpose requires, and applying full HIPAA safeguards including encryption, access controls, and audit logging. Developers should also account for state consumer health-privacy laws, which increasingly cover health data that falls outside HIPAA, particularly for consumer-facing wellness apps.

How long does it take to build and launch an FDA-compliant PT app?

There is no single answer, because it depends heavily on whether the app is a regulated device, the complexity of its features, the clinical claims it makes, and the integrations it requires. A wellness-category app can move relatively quickly, while a regulated medical device must account for validation, quality-system documentation, the FDA submission itself, and a second layer of app-store review. The most reliable way to compress the timeline is to settle classification early and engineer compliance in from the start, since the biggest delays come from discovering regulatory obligations late and having to rebuild around them.