“The Spin You Put on It…”
A
At the Holiday Inn, Hugh Calkins found himself at the center of a pecking party. His presentation of Cardima’s REVELATION catheter ablation system had not gone over well.
The chief argument he made for the REVELATION system was that it had to be better than the current alternative to the drag and burn, which was pulmonary vein isolation, the very procedure that he’d let Richard Wu practice on my wife 14 months earlier.
Hugh Calkins told the FDA that pulmonary vein ablation, which he was promoting and practicing at the time, was neither safe nor sound. At the same time that he was recruiting patients for the procedure on his website, advertising a proven and effective therapy that cured Dave Erdman the mountain climber, he was telling government regulators that “the safety and efficacy of pulmonary vein ablation was unknown then [in 2000] and it is unknown now.”
It seemed extraordinary to no one in the room that a doctor was telling the Food and Drug Administration, an agency designed to safeguard the public, that he has been performing a procedure which he considers to be dangerous and unproven. Calkins told the panel that cardiologists needed the REVELATION system so they’d have a procedure to perform while trying to figure out if pulmonary vein isolation was worth the risks—risks that Hugh Calkins was willing to take with other people’s lives.
“What’s nice about the REVELATION procedure,” he said, “is that it’s not a moving target. Here’s three lesions which can be understood and are reproducible. It’s clear how to deliver them. Where it fits in for a center like mine where we can do whatever we please, we’ll tell the patients that they have an option. If they want to go for the home run, the cure, and they’re willing to accept a procedure with higher risks that’s in evolution and so forth, we’re happy to go ahead and do a pulmonary vein isolation.” But using the REVELATION, he said, would be “a better way to be approaching atrial fibrillation rather than just doing a high risk procedure right from the outset,” which was the way that he was doing it.
The procedure he performed on Pam, or rather the procedure he let a rookie perform on Pam, was especially risky to the patient because it was a difficult and complex, and it was hard for doctors to get the hang of it. “Anyone who is doing pulmonary vein ablation realizes there is a learning curve and the learning curve is very rocky as you go up on it and the complications are like no other procedure that’s ever been done in an EP lab,” he said.
∞
Having explained to the FDA panel how he’d helped to create a need for the product, Calkins went about his main task of vouching for the integrity of the REVELATION investigational study. He put his Johns Hopkins reputation on the line and announced that “This study was very well designed, very well carried out, shows very sound data and very excellent safety.”
His colleagues on the Food and Drug Administration’s Circulatory System Devices Panel didn’t see it that way.
The FDA medical director on the panel, Dr. Leslie Ewing, started things off with a revelation of his own. Information had been leaked to him about what would turn out to be a relatively minor transgression: The study had been rigged.
Calkins & Co. were trying to put one over on the FDA.
According to the rules of the the investigation, patients were not to be told how many incidents of Afib they would have to experience in order to qualify for the study. Calkins had just informed the panel that ”the patients were not told they had to cross a certain bar to get in. “The thinking was that Afibbers, being offered what they thought was a chance of being cured, might exaggerate their symptoms in order to be eligible for the investigation.
Dr. Ewing announced that contrary to what Hugh Calkins had just told the panel, “It is the FDA’s information that the patients were aware that a minimum number of episodes were required.”
∞
Then Leslie Ewing read a laundry list of Hugh Calkins’s sins against science:
The primary investigators contaminated the data. This was a clinical trial to determine the safety and effectiveness of Cardima’s catheter system. But, for one particular aspect of the procedure, Calkins told doctors that all they had to do was try using the Cardima catheter first, and then they were free to use whatever type of catheter they were most comfortable with. Which was a bad enough breach of the study protocol, but there was more: Dr. Christopher White, head of cardiology at Ochsner Heart and Vascular Institute in New Orleans, pointed out to Calkins that “You have a whole bunch of patients in here who weren’t tried with your catheter at all.”
Further, the data from the study was improperly collected, so that there was no way to even know whether the patients actually received the ablation treatment.
There was a lack of rigor and discipline, investigators did not play by the rules. Calkins’s presentation did not even rise to the level of a true clinical trial; the way it was set up and run, it was merely an observational exercise.
Finally, Dr. Ewing said that because there was no way to determine the risk-benefit profile of the REVELATION system, the panel had determined that “from the clinical and statistical perspective, it is not clear if data can support any conclusion about the safety and effectiveness of the investigational device system.”
It was an outright rejection due to incompetence, and more humiliation followed as each panel member chastised Hugh Calkins in turn.
Sharon-Lise T. Normand, a professor at Harvard Medical School, holds a doctorate in biostatistics from Toronto University. Zeroing in on the data, and the conspicuous lack thereof, she kindly affected a hint of dumb blonde. “I just look at the numbers and I keep getting different numbers,” she said to Dr. Ewing. “Can you just go over it one more time?”
Dr. Ewing did his best, but the Cardima study was such a Rubik’s Cube that even a world class statistician piled high and deep couldn’t figure it out. ”I just want to get a sense of why we’re missing some data for the primary endpoint,” Dr. Normand said. “I know maybe the sponsor can tell me. Is it 87? Is it 83? Is it 81 and whether or not the 20 or 22 depending on how you count people…?”
She asked Calkins. “Statistically you also need to adjust for the fact that you have multiple centers. So you didn’t adjust?” Calkins said no, he hadn’t, adding “I’m not a statistician”—which was becoming more and more apparent.
For a research scientist, Professor Calkins appears to have a somewhat breezy attitude toward statistics. At a breakfast symposium hosted by Cardima in 2001, he explained how he crunched the numbers from Phase II of the trial. “The results of the first 48 patients can be interpreted based on what type of person you are —an optimist or a pessimist,” he said. “If you’re a pessimist, you’d probably say, `47 out of 48 patients had atrial fibrillation again, so it doesn’t work… ’”
∞
Dr. White was blunt. “My problem is that if you’re sloppy enough to do the protocol this way then I think that runs through the whole study. That contaminates the study. You wrote the protocol and then you didn’t stick to it. I don’t like that. I don’t quite understand the freedom of your investigators to deviate from the protocol. What was going on in their minds? These would be protocol deviations that would come up as red flags right off the bat. I don’t know why the waters were allowed to get so muddy.”
Dr. David Schwartzman, a New Yorker, knew medicine from learning in the trenches at NYU. He was not impressed by Ivy Leaguers. “I’m concerned that we’re approving a protocol or a technology and technique that is going to end up being a waste of the physician and patient’s time and money. My own personal experience with right atrial linear ablation is that only 10 percent of these patients remained without another procedure. So at the end of the day the long term benefit is hard to find,” he said.
Actually, Hugh Calkins generally agreed with Schwartzman. He did not believe in the procedure either. He published study results in 1999 that proved the procedure was worthless. When asked by an attorney in 2009 why he did not offer right linear ablation to Pam Walter instead of pulmonary vein isolation, he said the procedure “was not producing acceptable success rates.”
But he was doing Cardima’s bidding that day.
In addition to believing that Calkins’s theory was a waste of time and money, Dr. Schwartzman could see what was coming down the road if the scheme were approved. It seems that certain doctors, who shall remain nameless, were prone to be carried away with the technology at the expense of their patients. ”I have concern regarding the safety of this technology and its inevitable use in the left atrium as soon as it gets into the hands of those of us who have our own visions about this technology,” he said. “We get away with a tremendous amount in the heart as it is …” He spoke about “our arrogance about pulmonary veins and left atrial ablation,” the very procedure that Hugh Calkins had been experimenting with of late.
And Dr. Schwartzman was not alone in his feelings. A growing number of cardiologists began to believe that the technology was driving the treatment at the expense of patient safety. The American College of Cardiology issued this statement in 2002:
“It remains increasingly critical that the practicing physician acquire and maintain an understanding of relevant first principles of Electrophysiology. Although it is exciting, it should be kept in mind that the technology facilitates the application of those fundamental principles of Electrophysiology only for the benefit of arrhythmia patients.”
It was a message that was lost on Hugh Calkins.
∞
Unlike the rest of the panel, Dr. Schwartzman had little to say about how the study data was collected and organized. ”I respect the statistical mischief here,” he said. In other words, we all know what goes on, but Hugh Calkins, you have gone beyond decent.
Dr. Cynthia Tracy runs the cardiology shop at George Washington University Hospital. She thought the whole enterprise should be scrapped. “This study has many serious flaws,” she said. “I don’t think there’s anything salvageable.” One of the flaws she cited was the deliberate under reporting of adverse events along with the overstatement of the success rate. “It appears that the sponsor is using a narrow interpretation of the standard definition of adverse events,” she said, noting that had the rules been followed, the rate of adverse events would be 19% instead of 4%.
The success rate would have been 20%, not the 85% that Hugh Calkins claimed.
“The spin you put on it I understand and I’d love to believe you,” said Dr. Waldo.
Dr. Waldo could not believe the spin because he was a doctor and he knew better. But what if you were a trusting patient who fit the profile for a technique that a research doctor was experimenting with?
