Please Note: Blog posts are not selected, edited or screened by Seeking Alpha editors.

Lessons From Airline Industry To Dr. Scott Gottlieb - New Head Of FDA

Summary

Dr. Scott Gottlieb was announced as the new head of the FDA

Gottlieb’s goal is to speed vaccine approval and to streamline the FDA

This paper discusses concerns with the Gottlieb approach (potentially sacrificing quality for the sake of speed of FDA approvals)

This paper offers a series of proposals that emphasize lessons learned from the airline industry in the areas of product maturity, quality, and test that Gottlieb can use

This paper proposes that Gottlieb establish a board and standards committee to help make appropriate decisions

Introduction

Only 6.5% of the population has a fear of flying on an airplane and yet an airplane is quite literally a miracle of science – suspended on air thousands of feet above the ground.   Why is there such trust in airplanes if it can be dangerous?  Because airplanes have great safety records that have carefully measured data governed by the FAA and because airplanes have extensive testing on designs with over one billion flight hours in a 50 year period.  At the heart of airplane design is the ISO 9001 and AS9100 quality system which emphasizes risk management, product safety, requirements conformance, and continual improvement with a safety focus.  As airplanes become modernized there is a growing need for complex software and with that complexity is a need for software quality by utilizing the Software Engineering Institute (SEI) principles.

This article discusses how the healthcare industry can learn much from the airplane industry as the FDA moves forward with a new head, Dr. Scott Gottlieb.  Dr. Scott Gottlieb is faced with several problems which the airplane industry solutions can help solve within the healthcare industry space:

  1. How do you speed development of new healthcare solutions while lowering costs?
  2. How do you ensure safety of new healthcare solutions while moving more quickly?
  3. How do you ensure that new healthcare solutions are not only safe but effective in the shorter timeframes for development?

What is the current problem with the FDA we are trying to solve?

Per the US Department of Health and Human Services (NYSE:HHS) the major obstacles to conducting clinical trials in the United States include: high financial cost and lengthy time frames per here.  Per the same source the cost of clinical trials (without marketing) can vary but are usually tens of millions of dollars for a therapeutic solution.  Including marketing and all costs it can reach between $161 million and $2 billion to bring a new drug to market with an average length of 7.5 years.

The problem statement is that there are a lot of great ideas for healthcare therapeutics or preventatives but those great ideas do not reach the patients that need them due primarily to cost and risks associated with a long timeframe and being out a lot of cash over that long timeframe.  Many small businesses can simply not support the high cost to do business over such durations such that only the big players can succeed that have adequate assets.  Another inherent issue is that there are many patients who could benefit from great ideas in medicine today but the great ideas cannot reach them in time due to the lengthy approval process.

What can we do to overcome these problems and what are the risks?

New FDA head Dr. Scott Gottlieb’s goal is to speed development from the average length of 7.5 years to drive down healthcare costs.  That is a relatively obvious approach at the problem statement.  By reducing the time spent in clinical trials you can:

  1. Get great ideas to patients that need them faster
  2. Get companies with large outlays of money for clinical trials to recoup the cost of the clinical trials faster (thus allowing great ideas from smaller companies to reach the consumers and provide more consumer options)

But there are also risks with speeding development:

  1. How do you ensure safety thru speedy development?
  2. How do you prove that the great idea going in to the FDA is not really snake oil in disguise?  In other words, how do we prove a particular great idea works as intended if given a shorter timeframe for development?

As of yet Dr. Scott Gottlieb has not revealed how he will control these risks.  And Dr. Scott Gottlieb has not yet setup a board of people to oversee the changes at the FDA to discuss the approaches of reducing these risks.  News stories to date simply reference ideas from Dr. Gottlieb himself without representing where he is getting feedback (if any) on the ideas that he proposes.  That should be alarming to those that are following the stories because the process of controlling these risks needs much definition and much discussion.  Without a board to help support Dr. Gottlieb the ideas of speeding development may sound good on paper but may eventually fail due to lack of detail as to how the risks will be controlled.

The article that you are reading will attempt to provide ideas on how the airline industry lessons learned can be utilized by Dr. Gottlieb as a “template” by which the FDA can act.  But Dr. Gottlieb cannot do it alone.  He needs a board of people to listen to his plan and offer feedback.  Until that happens we should have a level of concern that the risks are not being addressed adequately to support the high level goal of speed of development to reduce healthcare costs.  Dr. Gottlieb’s first step should be to announce knowledgeable board members to help him with the details.

What should be the emphasis of the proposed Gottlieb board?

The airline industry teaches us that risks are acceptable thru proven design, quality emphasis, and test. 

At the cornerstone of speed (which is Gottlieb’s intent) should be a focal point on proven design specifically.  In the airline industry second generation aircraft based on proven designs can typically take 50% of the time to release to the public than first generation planes.  That is a significant speed savings over less proven first generation models.   As an example, the 2nd generation Boeing 777 took 4 years and 2 months compared to a 1st generation Boeing 787 which took 8 years and 9 months.  The speed in this case comes from reusing parts that are already proven to work within the industry.  In a similar way biotech can reuse portions of designs that are already proven to work in the industry then those designs should be able to advance more quickly with fewer risks than biotech designs that use 1st generation parts.  This “maturity design reuse” model can help to achieve Gottlieb’s goals of speed and help to ensure safety.  The proposed board members for Gottlieb should fold maturity of design in with speed based decisions.

For cases where biotech has to use 1st generation parts to its design speed becomes more difficult and that is where quality measures must form the secondary cornerstone to obtain Gottlieb’s goal of speed.  Quality does not need to be invented or reinvented, however.  Instead, existing quality techniques should be emphasized by the proposed Gottlieb board members.  As an example, the FDA has already implemented Quality by Design principles in small portions of the biotech industry.  It is my fear that Gottlieb may tend to steer away from these quality principles which in my opinion would be a mistake.  Instead, I would propose that a Gottlieb driven board emphasize Quality by Design (QbD) in every stage of planning for a new drug to market.  It should be noted that QbD does not occur only on approval but it is integrated in to every step of planning the drug from defining the opportunity all the way thru testing of that opportunity and even in to the delivery to the customer.  Further, QbD does not stop there but also defines an optimization phase whereby there is a feedback loop to control variation of expectations of a particular drug to actual experience.  That optimization loop refines the product and puts in “product stops” in the event that the control variation between expectations and actual experience is significant.  For example, I would propose that if a particular drug would have an expectation of “fewer than X% SAE’s (Serious Adverse Events)” and the actual experience is greater than X% that the drug have a stop be placed on it.  Those thresholds and metrics to collect for what defines a product stoppage and what doesn’t should be controlled thru the proposed Gottlieb board members and it should be layed out in a product plan well ahead of product shipment.

If we look at the airline industry it heavily utilizes optimizations and iterations as part of its quality plans.  As an example the airline industry and SEI quality principles utilize the Capability Maturity Model which at its most mature levels drives “Efficient” levels that contain “deliberate process optimization/improvement”.  It is these kinds of “optimization feedback loops” which should be built in to the Gottlieb board plans if Gottlieb’s goals of achieving speed with quality are to be obtained.

The FDA can further learn from the “scrum process” which emphasizes quick turn iterations of small well defined scope (typically in two week durations) to provide quick turn iterative feedback to “product owners” as to quality and design goals.  These quick turn iterations in scrum are called “sprints”.  As an example, a typical clinical trial phase 1 historically lasts one year to complete for roughly 40 to 80 patients.  Many times the companies are blinded to data until the end of the 1 year duration.  That 1 year period of time is lengthy and counter to Gottlieb’s goals for speed.  But what if we designed a phase 1 trial like a “scrum” with quick turn iterations (aka “sprints”)?  For example, let’s say that we defined a phase 1 like this:

Sprint number

Duration

Inputs to sprint

Outputs from Sprint

Go/No Go criteria

1

2 weeks

  1. Patient seems to meet inclusion criteria
  2. Patient first dose
  1. Adverse Effect Metrics
  2. (perhaps even prior to sprint 1) Genetic scan of patient recording potential biomarkers – import to cloud for future iteration optimizations
  3. Initial immune response – import to cloud for future iteration optimizations
  1. SAE’s < X%
  2. (perhaps even prior to sprint 1) Do exclusion criteria rule out patient based on genetic information?
  3. Endpoint titer or similar measurement greater than expected threshold

2

2 weeks

  1. Patient second dose
  1. Adverse Effect Metrics
  2. Measure immune response - import to cloud for future iteration optimizations
  1. SAE’s < X%
  2. Endpoint titer or similar measurement greater than expected threshold

X

2 weeks

  1. Final metrics collected and documented – import to cloud
  2. Patient follow up

Final conclusions to endpoint expectations

Endpoints met?

The idea behind these sprints is to give immediate and frequent feedback to the product owner (where the “product owner” is usually a principle investigator or the FDA itself) as to whether or not the trial should continue.  All of the “blinded” concepts used historically are removed in this sprint process in order to achieve Gottlieb’s goal of speed while at the same time preserving quality.  A company may find 2 weeks in to a trial that it is time to stop, for example, rather than proceeding an entire year without knowing that they should have stopped at week 2.  On the flip side if a particular drug is required ASAP by the FDA to meet a particular consumer need and the sprints are progressing well then the FDA may choose to move a particular drug to a phase 2 without the need to accomplish all “X” sprints.  In this way each sprint represents a potential decision point for the FDA and for the company/Principle Investigator rather than having the decisions wait the entire year which was the historical means.  It would be my proposal that the FDA board under Gottlieb consider such “agile” methods for determining when a particular drug should move forward or be stopped.  It does allow for speed but at the same time it puts in place specific measurable quality goals at each step.

Also if a particular sprint does not go as planned then it may be in the FDA and principle investigator’s interests to redesign the trial for the next subsequent sprint right in the middle of the trial.  This “agile thinking” would be completely new to the FDA but the idea here is that if you have a design and expected outcome that did not go as planned you would have the ability to dynamically change the trial design for the next “sprint” (perhaps even mid-trial) to try to achieve better results.  The scrum process fully supports this concept by allowing each sprint to be a design cycle in and of itself which allows for dynamic changes as the sprints continue.  In this way, perhaps the inclusion criteria for the trial might change at sprint 3.  Or perhaps a dosing level might change at sprint 4 of the trial.  These dynamic adjustments would lessen the need for a “phase 3” to determine appropriate dosing, for example, by building in the amount of dose required as part of each sprint.

Whenever we discuss quality we should at least touch on the concepts of DFMEA and DMAIC since they are utilized successfully in the airline industry to help offer safety thru quality techniques.  That leads to the following two paragraphs which the Gottlieb administration would be wise to include in the requirements for product approvals.

Associated with any technology product (such as a biotech product or an airline product) is a set of identifiable risks.  These risks should be tracked and constantly reviewed by the product team.  One way of doing that tracking and review is thru a “Design Failure Mode and Effect Analysis” or DFMEA.  The idea here is to review the design and brainstorm the potential failure modes that can exist in the design and then track/rate those risks thru an RPN calculation that is weighted based on potential severity of the risk versus the potential of occurrence versus the potential for detection.  Just to give an example, in the airline industry there is a risk that an engine motor might fail.  In the case of an engine failing the RPN would be weighted heavily towards severity (an engine failure is severe) but the RPN should have a low rate of occurrence (hopefully) for an engine failure.  From there a risk manager might think about the third form of RPN weighting (detection) to see if there is sensor data and metrics that can be collected to detect the potential for engine failure prior to it becoming a real issue.   In this way there might be a set of product requirements placed on “detection” to address the risk of a particular design failure mode (such as engine failure) which may not have been in the set of requirements if the product had not gone thru the DFMEA process.  In a similar way DFMEA can be applied to vaccine or drug design.  For example, if a particular drug is administered via injection is there a set of ‘detection’ requirements that are needed for the product in order to reduce the risk that the vaccine was not delivered correctly?  The list of risks should ideally be documented by the company and presented to Gottlieb’s board prior to the trial design being approved by the FDA to ensure that the company has done its job with respect to analyzing potential failure modes and addressing those failure modes prior to beginning.   This exercise really does not take much time and in the end can help achieve Gottlieb’s goals for speed by reducing the number of actual failure modes thru a review cycle of potential failure modes early on in the product lifecycle.  In other words, a failure mode caught and fixed early in a product life cycle will lead to approval of that product quicker than if that failure mode is caught and fixed late in a product life cycle.

There is another industry wide process which Gottlieb can utilize easily in to the FDA which is the DMAIC process from six sigma.  The DMAIC process emphasizes an acronym of “Design, Measure, Analyze, Improve, and Control”.  The idea here is that it is not enough to simply design a vaccine but you need to measure the metrics associated with that vaccine, analyze the effects of the vaccine, constantly improve upon the design of the vaccine, and control the variances of expectations to actual results.  The DMAIC process is used within many companies successfully already and would be easy to fold in to the Gottlieb requirements of approvals.

You will also notice in the above table that we have references to “import to cloud for future iteration optimizations”.  This is intentional to address the “optimization” and feedback loops inherent in quality measures such as the Capability Maturity Model, the Quality by Design (QbD), and scrum whereby the goal is to try to continually improve.  The idea here is that there is a lot of data collected during a clinical trial.  That data should not go on paper because paper is not easily searchable.  Rather, that data should go to a database on the cloud whereby algorithms can crunch that data and determine better approaches for future iterations of addressing a particular infectious disease or cancer treatment.  The idea is to use the data that you have to make your future attempts more successful rather than burying the useful data somewhere that is not useful to other attempts at solving the problem.  As part of that data storage to the cloud there may be some data that could provide a competitive advantage and there should be rules placed on such data as determined by the proposed Gottlieb board.   For example, certain data within the cloud may be marked as “owned by company X” such that if “company Y” wanted to use a design that leveraged data from “company X” then a royalty fee must be payed by company Y to company X.  Those types of issues would need to have rules associated with them as determined by the Gottlieb board but the main point here is to continuously improve based on “useable data” where ideally that “useable data” would exist in the cloud.

And that brings us to the third cornerstone of emphasis by the proposed Gottlieb board for 1st generation designs which is test.  We need to test and prove that a particular drug is not snake oil (it should work as described) and it should be safe before it is in widespread use.  Further, the consumer is going to be confused with a plethora of drug options being approved by the Gottlieb administration (since Gottlieb intends to approve more drugs faster) such that the consumer should have searchable access to the test data that is collected.  For example, if a patient has lung cancer and the Gottlieb administration approves 100 drugs for lung cancer then the patient should have at its finger tips a searchable database of metrics associated with each of those 100 drugs (ideally thru the cloud) including immune responses, a list of adverse effects, how mature the product is, and most importantly the results of patients taking the product.  That data needs to be coming from a database that is accessible by the consumer rather than coming from a doctor who may have been compensated to suggest a particular option to the patient.  In other words, the patient should feel empowered to make their own decisions based on the data available.   The interface to this data should have an open interface such that “apps” can be made available to help consumers with the choices of available drugs since under the Gottlieb administration the choices may be confusing without those apps.  The apps should be able to summarize the list of standardized metrics from the approved drugs and offer recommendations from that data.  Having choice of which app to install by the consumer also lessens the chance that there would be favoritism involved with pushing or suggesting that a particular drug be used by the consumer.

But where does test fall in to all of that?  Well implied in the above paragraph is a need to standardize what forms of test data to collect.  Some standardizations are obvious such as a list of adverse effects, a list of side effects, suggested dosing, and results of using the drug.  However, other standardizations are less obvious.  Examples might include parameters of delivering a particular drug dose (if by injection – the injection methodology, for example).   All of these parameters need to be a part of the trial design upfront.  And Gottlieb’s board should have a standards committee which approves less obvious standards which would be a part of a new trial.  That standards committee can then help guide other trials to use some of the same database fields that other trials have used which, in turn, will help to keep the cloud database of test results standardized and straight forward.  That will help to lead to more searchable and useful results.

But test plays another role which is a set of metrics to be used as a ‘baseline’ for a particular methodology.  That ‘baseline’ can be useful to the FDA to compare a particular solution to other solutions used by its peers.  Just as an example, it is easy for one group to say that it has a Zika vaccine that works when in fact the Zika response is worse than its peers.  The FDA would not know the difference unless a ‘baseline’ for test metrics is established for peer comparison.  If we had such a baseline established then I would not have to write articles like this one explaining why the Inovio Zika vaccine is superior to the NIH Zika vaccine.  Instead, the comparison would be obvious because a ‘baseline’ would exist and the FDA could see at a glance that the Inovio solution is superior.

A technology product such as a biotech product or an airline product should have a list of requirements associated with the product.  The requirements dictate what a particular product must do.  If requirements are written for a product correctly then they should be testable.  Therefore there should be as part of every test plan a guarantee of “requirements conformance” by ensuring that the product meets the requirements as intended.  It is proposed that it become a part of the Gottlieb board responsibilities to review the “requirements conformance” checklist for each product approved and ensure that the test coverage is adequate to ensure that the product works as intended.  By doing so it should give the consumer a level of confidence that the drug that they are taking works per the original intent of its design.

It should be noted that test and quality should go hand in hand.  Any quality plan for a product would have a test plan.  That test plan should have the steps for test as well as the expected metrics and requirements conformance checklist for test documented.  The quality team should have a review cycle of that test plan and the quality team should be involved in the signoff of the test plan as well as a quality signoff on the test results.  Additionally it would be a good idea that if a trial is defined as a set of ‘sprints’ that there be a set of test results expected at the end of each sprint and that quality be involved in the signoff of each sprint at each step of the way.  Doing iterative design/test/results/signoff at each sprint will help to achieve Gottlieb’s goal of speed and hopefully avoid the pitfalls that I am concerned with for the Gottlieb administration which are the quality aspects. 

Conclusion

Several people are ‘thrilled’ by Gottlieb’s confirmation to head the FDA.  However, few details are given as to how Gottlieb will achieve his goals for accelerating drugs thru the FDA without risking safety issues or providing approved drugs that do not work as intended.  Currently consumers rely on the FDA thru trust that a medication prescribed to the consumer will work as intended.  Right now the biotech industry has a unique opportunity to streamline processes of the FDA and provide solutions to the consumer more quickly than in prior decades.  However, the FDA has to be careful to not violate the consumer trust.  If the FDA rushes products to market and violates the consumer trust it can create a consumer backlash that would set the FDA backwards by decades of progress.  This paper emphasizes lessons learned thru the airline industry thru already existing techniques of product maturity, quality, and test which the Gottlieb administration would be wise to incorporate to strengthen Gottlieb’s acceleration goals without sacrificing safety or product quality.  Let’s hope that Gottlieb does not sacrifice quality for the sake of speed but rather embraces quality as I have documented herein.

Disclosure: I am/we are long INO.