Does Medicine Need a Paradigm Shift?

Let's start with physics

As my brother's T-shirt says "The Pulse of the Earth is in Physics". Physics is a fundamental science. Also called "pure" science. That is a fancy way of saying it is reductionist. When you think of an apple falling to Earth in physics, all you think about is its mass and the forces acting on it. Everything else is immaterial to physics, including the questions like "Is the apple rotten/ripe?", "What is the probability of the apple falling on a rabbit and killing it?", "Are there hungry people waiting for the apple who won't get to eat it?", and "Is the apple cursed?"

The question whether apple is rotten can be answered by another branch of science called biology. Physics and biology are called natural sciences. These are branches of science which rely on observation of the universe to reach at inferences on how the universe works.

The question on probability would fall under mathematics. Mathematics is a bit different from natural sciences. Because it is based on axioms and logic. Such sciences are called formal sciences.

A hungry class of human beings not getting to eat apples and the reasons behind it would be the matter of study in social sciences.

The cursed apple is a subject of religion and superstition. These are, by definition, not questions for science to answer.

What kind of science is medicine?

Medicine is not a pure science like physics. It is an inter-disciplinary, applied science. Medicine uses several branches of science like biology, chemistry, and mathematics in its own goals.

A medical practitioner has to know several sciences like anatomy, physiology, biochemistry, pharmacology, and microbiology to be able to practice medicine well. They would also need skills in probability, reasoning, and logic. Also critical are skills like communication, empathy, leadership, and management.

There are also several other forces in play that influence the practice of medicine - education, medical training, health systems, politics, economics, religion, human resource, war, and so on.

The question of a paradigm shift in medicine is thus complicated. Which part of medicine would the paradigm have to shift in? In the numerous sciences that make it up? In the way it is practiced? In the way people are trained in it? In the way the systems around it are organized?

Science is the only way of knowing 

What is science? 

The opening statement from Wikipedia is: "Science is a systematic endeavor that builds and organizes knowledge in the form of testable explanations and predictions about the universe."

Science is what allows human beings to operate in the world. It is the sum total of all that we know about the universe through thousands of years of living in it and observing it. It is the reason why we know that if I strike a lighter in a particular way with a knob turned on the gas from a cylinder will come into the stove and start a fire. It is how we cook and eat. It is the reason why we know that elephants can lug trees while cats or dogs can't. It is the reason why we are able to talk to each other over the internet.

Everything we know about the world is through theories and observations that confirm those theories. When we come across observations that contradict those theories, we are forced to come up with better theories. But till then, we seem to be able to live on earth with the old theories.

Is there any other way of knowing about the world? Think about it. Everything that you know about the world would come from your own observations and theories, or those by others that you have read about. There is simply no other way to know facts about the world.

You might say, "Oh, to know whether it is raining, I just have to look out of the window. No science involved". But hey, what you're doing is observation. And then forming a theory that it is raining. What if there is a film shoot going on and they're pouring water with a hose and that is what you're observing through the window?

The whole experience of seeing water drops falling down from sky and knowing that "it is raining" is based on science. It is based on human observation since time immemorial of the natural phenomenon called rain. Even when you're looking out of the window to say whether it is raining, you're using science. And it is science that allows you to say whether it is actually raining or a film crew pouring water.

You might also say, "Hey, I know cycling, is that science now?"

When you say you "know" cycling, the knowing refers to a particular sense of muscle memory that you have developed through practice. But this is not the kind of knowing we are talking about. We are talking about knowing how the universe and everything in it works.

Read a related post about this question of whether science is the only way of knowing, where I argue that if there is a way to know, then science is the only way of knowing. Consequently, there are some things we cannot know, and this question would not apply at all.

Queering science

While indeed science can be seen purely as methods of rationality as above, it is has to be acknowledged that science is ultimately a human endeavor and thereby it reflects all the faults of the human society almost as it is.

I've dealt with this human aspect of science in an earlier blog post and so I won't repeat those points here. Suffice to say, there is an intersectional approach to the practice of science that's missing in mainstream science.

What about applied sciences?

When it comes to an applied science like medicine, the problems seem to compound. Many of the sciences that make up medicine are all super hard to study. The tools we have are limited. And the institutions that we have are very problematic spaces (in terms of patriarchy, violence, oppression, and discrimination).

When faced with such a complex challenge, many people prefer to run away and find comfort in places that nobody is finding faults with (although they would be riddled with even more issues). That's why many people turn to Reiki, Homeopathy, Ayurveda, and so on. This gives them psychological comfort. But this is no solution to anyone's problems. We will talk about that later.

Applied sciences deal with the real world. One that is filled with uncertainties. One where perfect knowledge is impossible, but action is inevitable. It takes a lot of interdisciplinary thinking to operate in the field of applied sciences.

Let us look at what some people call Evidence Based Medicine. EBM is misunderstood by many. They give undue stress to the word "evidence" and think that a randomized control trial is the be-all and end-all of EBM. These are the people who assume that medicine is based on a paradigm of large numbers. What they do not know is that there are three pillars of evidence based medicine:

  • Clinical judgement
  • Relevant scientific evidence
  • Patients' values and preferences

Clinical judgement is where the practitioner comes in. The validity of medicine rests on the practitioner making the right observations and judgements about a particular situation. Similarly, we need a body of evidence, a body of science before us to be able to make any intelligent observations. And considering all of this is about a patient, it is imperative to keep their preferences in the whole matrix of evaluating what to be done.

Let us talk about relevant scientific evidence a bit more because that seems to cause a lot of confusions in the world. (Even in an otherwise brilliant talk about integrated medicine, Ravi Narayan equates medicine to controlled clinical trials, for example. (19:30 in the video)).

It is all about knowing the truth, as we discussed in the beginning. How do we know what to do in a particular situation. When someone comes in front of you with cough, weight loss, and fever, what do you do? What if you also found in the sputum of this person the organism that is known as Mycobacterium tuberculosis? What do you do? How do you know what to do? That's the important question.

If you knew magic, you could perhaps try that. You could get rid of all the M. tb from their body magically! That would help them. You might save them from certain death.

But if you didn't have the evidence built over centuries of human beings struggling with this disease called tuberculosis, how would you even know that this person would die soon?

It is only the scientific method of knowing the universe that can guide us to move even an inch forward towards helping those who are struggling.

The alternative to science

The alternative to science is the pandemonium of opinions and beliefs. There are people who consider these as ways of knowing the universe. But they don't critically think about their own philosophy.

Firstly, whose opinion counts? Who is authorized to make opinions? Is it reserved for people who meditate in the Himalayas? Can you and I do it? Does it have to be done high on weed? How do we measure whether someone is legitimate in claiming that the shit they pulled out of their ass is the correct knowledge about the world?

Secondly, when you have two people claiming two shits that are contradictory to each other, what do you do? Let's say person A says eat leaf A, while person B says eat leaf B when confronted with the patient we saw above. Which leaf should the person eat? Both leaves? No leaf?

The only way to evaluate anything and arrive at an actionable prediction about the universe is through science. If you look at what's typically called pseudoscience, things like Homeopathy, what you can see is that underlying all these are certain theories that are of very low quality. These theories are sometimes not verifiable. And if at all they're verifiable, they end up to be false. Proponents of these pseudosciences typically take comfort in the space where they come up with a theory, believe in that theory, and don't bother verifying those in the real world.

Paradigm shift in medicine?

Having said all that, let us come to the question of the need for a paradigm shift in medicine.

It is easy to speak in vague terms about "holistic" approaches that incorporate a paradigm of being "more rigorously attentive to the individual while keeping in view the larger picture". But when it comes to practice, we can quickly see how rhetoric like these are hollow.

Does attentive to the individual mean using genetics and personalized/precision medicine? Does it mean just taking patient preferences into consideration? How scientific and rigorous do you have to be when you say "rigorously attentive"? If a person says "I think homeopathy will work for me" and you diagnose tuberculosis in them, what do you do?

What about the other question. How many people practicing modern medicine are actually practicing evidence based medicine? How many do rely on science and evidence to manage their patients? How many randomized control trials did people use to prescribe drugs during COVID. How many RCTs are followed when people prescribe platelets and antibiotics for dengue? How many RCTs are followed when people diagnose typhoid with a single Widal test of 1:40?

Does the "larger picture" include social, political and economic determinants of health? But does it also mean that the focus should only be on distal determinants? Would you not worry about Anti-Tuberculosis Therapy in someone with TB or will you only keep saying "nutrition!", "nutrition!", "nutrition!". Fine, nutrition. But how? Will you feed this person out of your pocket or will you keep saying the government should come with food security schemes? Fine, the government should come with food security schemes. But will you work with policy makers on making such schemes a reality or will you keep writing about it?

Yes, a paradigm shift is necessary. A paradigm shift that puts people first. A paradigm where sacrificing rationality for practicality and/or sacrificing science for pluralism doesn't kill innocent people. A paradigm where working on social determinants goes hand in hand with treating now those who are suffering now. A paradigm where paternalism and saviour complex are replaced with solidarity and praxis. Nobody can say no to that paradigm shift.

***

Footnote: There's a human tendency to come up with alternative hypotheses to explain seemingly miraculous phenomenons. When I was 16 years old, I came up with "ASD rays" to explain telepathy. Thankfully there was a group of people who explained to me that my theory, however "sound" explains a phenomenon that's non-existent. At that point in time James Randi offered a million dollars to anyone who can demonstrate paranormal claims. And nobody won it, of course.

As long as people think that things like homeopathy actually are more than just placebo, they'll come up with theories that go into sub-atomic realms to explain how these work. That's natural. And they'll keep struggling to understand why rational people reject their theories. If you are empathetic to them, you'll realize that to them it is inevitable that these theories must be true because otherwise how do they explain to themselves their "miraculous cure" that others believe is charlatanry?

By Doing "Government's Work", Are We Making It Easier for The Government and Worse for the People?

At the end of the CHLP session today Akshay (not me) asked something like: "When we do work that the government should be doing, are we making it easier for the government in some ways, and also making it more difficult to hold the government accountable?"

This is a question that only someone who is truly invested in community work can ask. They are worried that the government is going to invest less in that particular problem, that in the long run it becomes harder and complicated because of the reliance on "bespoke" solutions. (The example given was how government relies on the voluntary effort of data by covid19india.org / covid19bharat.org to get COVID related counts and how there is no other system to track these counts)

I do not claim enough experience to answer this question.

But if we break down this question, the concerns we have are:

  1. How sustainable are such bespoke solutions? If we could keep doing it forever, then why should we not do it forever? Should government ever take over?
  2. Are such bespoke solutions less effective than more universal solutions? If yes, are we causing a less than optimal outcome? If no, are we preventing a scale-up of these solutions by the mere fact that it came from outside the government?
  3. Does access to and/or existence of such bespoke solutions make it difficult to demand more universal solutions from the government? (Either by making people reticent or by making the demand look less urgent)

A few counter points are:

  1. But how long should I wait for the government to do the right thing?
  2. Who is at the receiving end of our desire to wait for a universal solution? Who suffers when we wait?
  3. Let's say I don't attempt the bespoke solution. What do I do now? Should I now force the government to build a solution?

The way I avoid these questions are by thinking:

  • The government is a huge, inefficient, highly hierarchical organization with not much capability to build innovative solutions. Therefore, expecting government to come up with a good solution is pointless.
  • I should do things that give me joy, not what brings joy to the world. If bringing joy to the world in certain ways brings me joy, then so be it.
  • The second-order, third-order effects of our actions are very very hard to predict. No matter how much we "calculate", not much is going to come out of the calculations. We have no way to say that any particular action is what is going to help the world. We just do what we want to do and hope that it turns out to be a good thing. Often, there is no way to actually say whether something turned out to be good either.
  • If we are creating value, putting value out into the world, it is more likely than not that we're doing something right. The value will compound in ways we cannot anticipate. Always.

 If you are reading this and you have answers to some of these (existential, sorta) questions, let me know.


Update

I sent this to Tanya and Prashanth. Prashanth tried to add a comment and failed. That comment is:

"This is an important question to "struggle" with especially for those (like me) who are involved in such "solutions" that are often not only outside-the-box, but also as rightly pointed out, being designed outside the "public" system. For an individual like me for whom, working with indiviudals/communities/populations is coming from an ethical imperative and from wishing to move our society towards health equity, there is - I confess - no other way. What do we who do not wish to work within governments for various reasons do? I think what we can do is build coalitions, networks and allies which nudge/push/critically demonstrate the need for public services and systems to do more. And for me, such efforts are ways of showing that more can be done. Another reason to do this is to address the inertia that sometimes develops at middle level institutions (like districts) where the glamour of word/jargon based policy vocabulary is not there and the fatigue of under-resourcedness is a daily reality. So, I believe such efforts can hopefully spur creative thinking within public systems, build allies within the system and who knows...knowing the complexity and unintended effects these things have...some things stick...some things flourish...improve? But, certainly there ought not to be a claim that such accomplishments (if they are such) will automatically result in "systems change"....these are some of my thoughts. "

 

Prashanth also got Werner Soors involved. You can read W's comment below this post. To me, W has more or less arrived at the crux of the dilemma. The struggle is related to the dichotomy created by the ideal government and the real government. But as W points out, it maybe worth trying to become part of the government through becoming part of the people.

Coincidentally, I saw this video by The Ugly Indian today



What Can An MBBS Doctor Do?

In the protest surrounding suspension of Dr Saibal Jana and Dr Dipankar Sengupta, a debate has emerged around what an MBBS doctor can and cannot do, especially in rural settings. This is a very complex question that requires a complex legal answer.

There are several relevant case laws and even acts like Clinical Establishments Act which talk about some aspects of this debate. But let us look at it from a more fundamental and fresh perspective.

Law is not static. It is subject to continuous change. Law is not blind either. It is acutely aware of context. Therefore, there is no need to frame a universal, absolute, strict law regarding a nuanced question like this.

What are some of the considerations that must be kept in mind when framing a law on this question?

- How to bring equitable healthcare to the people of our country?
- How to protect people from harm?
- What is the situation with respect to human resource availability in rural healthcare?
- How do referral pathways work in our country?
- How does medical education work?


What makes a rural place "rural"? Places are considered rural when they have small population and consequently very few markers of urbanization (like large buildings that accommodate many people in a small area and huge roads that accommodate heavy traffic). Many rural areas won't have a movie theater. Because there are so few people that it would be difficult to run a cinema and make profit. Similarly, the economics of small numbers do not allow a "specialist" doctor to practice only their specialty in rural area. It also makes it difficult for them to invest in equipment that might be required for specialty practice. In many ways, specialist practice is economically impossible in rural areas.

On the other hand, a generalist is able to successfully practice in rural areas. Someone who is willing to see a large number of people with many different health conditions can survive in a rural economy rather comfortably.

Is it possible to have multi-specialty hospitals in rural areas, if the rural economy cannot sustain specialists? Yes! This is possible through team work. There are many rural hospitals which work by association with specialists who might be present only on one day a week or available over phone calls. This unique symbiotic arrangement has organically developed in many rural places in India. The reason is that just because a place is small, the health needs of the people in that place will not be small. (To paraphrase Dr Yogesh Jain). Rural places also require specialized care. The demand is there, but the volume is low.

If you can have one specialist come on one day and manage all the cases that require that specialist's care, the rural hospital can club many patients together on that particular day and make it an economically feasible day for the specialist. If the specialty is something like surgery which requires post-op care and follow-up, rural hospitals can manage with generalists who work with the guidance of specialist in arranging that follow-up care.

What specialists typically tend to do in such arrangements is also empower the rural generalist in being able to handle more complicated cases. This happens in many ways. The availability of specialist guidance increases the confidence of the generalist. Doing things with a specialist transfers necessary skill. And working under these arrangements for a while makes them able to work independently as well.

That is how medicine is. Medicine is not something that you finish learning in a specific number of years in a medical college and then go out and practice forever. Medicine is something that you learn every day. Even the specialists learns on the go. They hone their skills day by day, with every new patient.

Now, let us imagine the same rural area without this delicate arrangement in place. Imagine a doctor who has just finished MBBS has come to practice in a rural area and have started a small clinic or are in a PHC. What are they supposed to do there? Can they treat pneumonia? Can they manage someone with schizophrenia? What about deliveries? Can they conduct a delivery? How about I&D for abscesses? Can they prescribe Morphine for palliative care? Would it be alright for them to stabilize a poly-trauma patient? Someone with an Acute Coronary Syndrome? What happens when a patient comes to them with long history of cough and fever? What about someone with chronic headache? How about someone with loss of balance? Or someone with a distal radius fracture?

In a world focused on specialties and urban model of care, many of these patients would have to be referred to the average specialist in the nearest urban setting. But there is not a lot of insight into how many of these referrals are successful. How many reach the right kind of "specialist"? How many decide to suffer than seek inaccessible care? How many settle with an alternative medicine practitioner who decides to take the risk of handling the condition with the knowledge and confidence they have? How many die lost in the referral pathways? How many die at home?

In a world that's person-centered, we would encourage the MBBS doctor to take all of these factors into consideration and take a calculated risk in cases where that would be in the best interest of the patient. In cases where the patient is otherwise going to not receive any care, it is often in the best interest of the patient that the MBBS doctor, even if they do not have the skill of an average specialist, attempt something risky. 

Of course that shouldn't come at a cost to the patient. This has to be a careful decision that's discussed with the patient. A shared decision has to be made between the doctor and the patient as to the risks and the alternative options. But it is these informed risk taking that's going to help that doctor level up. 

A progressive outlook at medical education should think about what resources can be made available to this isolated doctor to be safe in the risks that they're taking. What kind of guidance and resources can be made available to them to increase their chances of success and increase their level of competencies. 

It is when we are able to create such empowered generalists in rural healthcare that we can start bridging some of the huge gaps in rural healthcare. The law should not become an obstacle in this mission. The law should be progressive enough to encourage these possibilities. The law should be promoting this decentralization of healthcare. The law should be focused on people and their well-being.

The question should not be "What can an MBBS doctor do?". The question should be "What should an MBBS doctor do?"

Essential Digital Literacy for Community Health Folks: Part 1

Whether one likes it or not, everything is getting digitized. And it is often a good idea for human beings to keep abreast of changes. This is a series of posts designed with community health folks in mind to help them develop mental models around the technologies that make up the digital world.

In this post, we will look at certain foundational terms like "information", "data", "communication", and "computer". Then we will connect it to words like "internet", "server", and "cloud".

***

Information / Content / Data

Anything that is meaningful is "information". Emails, videos, textbooks, numbers, anything that you can imagine and represent or store in some form.

"Content" is just another word for information used in specific contexts. Like if I'm sending you an email, the body of that email would be called "content". An article has content. A youtube video has content. An instagram post has content. A tweet thread has content.

"Data" is yet another way of looking at information. If you collect information about 50 people while doing a research project and put it in a spreadsheet, you might call it research data. If a hospital keeps a medical record of a patient who was admitted there, that would be called health data. If you write a brief bio of yourself and share it with someone, it might be called a biodata. 

***

Communication

Human beings have been communicating forever. We can talk to each other. Or we can draw something on the wall which someone else can come back and read later - perhaps after a day, perhaps after centuries. We can write letters. We can write emails. We can message people.

Communication is just transfer of information/data from one place to another, from one mind to another.

It need not always be one-to-one. It can be one-to-many. Mass communication.

We will come back to the term 'communication' in a while.

***

Computer / Computing device 

A computer is a machine or a device which can be used to view, store, transmit, receive, and manipulate/transform data or information.

Is a physical book a computer? It can be used to view, store, transmit and receive information. But it cannot manipulate or transform that information.

What about a calculator? Is it a computer? A calculator can be used to view, store and manipulate/transform information. But it can't really transmit or receive information, can it?

What about a smartphone? You can send and receive data/information via smartphone. You can store it and view it. You can also manipulate and transform it. A smartphone is a computer.

So is a laptop, or a desktop.

Computer as a Communication Technology

You might have noticed that in the above section, I am referring to the computer as a machine that can be used in receiving/transmitting information, or, communication. In the past people might have called a calculator a computer. But today, computers are almost universally able to communicate and therefore it is ideal to view computers as machines useful in communication technology.

What kind of communications do computers allow?

Email, WhatsApp, YouTube, Instagram, Twitter, research publication, reading journals, reading news, writing blogs, reading blogs, putting things on a website, viewing a website, so on.

(Remember - your smartphone is also a computer!)

***

Internet

The Internet is the simplest and most powerful creation of human beings in the past few decades.

It is super simple. Imagine I (A) connect my computer and your (B) computer with a cable that can transmit information. Now I can send messages from my computer to yours and vice versa. A---B

Imagine now that you connect your computer with that of another friend (C). Now, I can send a message to C through your computer.  A--->B--->C

If D connects to C's computer, D can send a message to me. D--->C--->B--->A

Imagine most of the computers in the world connected to each other through each other. Like a huge "net". That's internet.

This connection need not be through a physical cable.

It can also be through the electromagnetic spectrum. 4G, 5G, WiFi.

You might have a question here. You have only one computer in your place, and it is not connected to any other computer. How are you able to browse the internet, then?

Well, actually, when you're connected to internet (be it through wifi, be it  through mobile data), what you're actually connecting to is a computer. That computer would be in the office of your internet service provider (Airtel, BSNL, Jio, etc). And they connect their computer to the rest of the world through massive underground cables.

Basically, the whole world is connected through cables and electromagnetic spectrum. And that's how internet works.

***

Server

A computer is not a magical device.

If your computer is switched off, you cannot read your emails from it.

If your computer is not connected to the internet, it cannot send or receive information from the internet. If your wifi is switched off, or your data pack is over, you cannot receive whatsapp messages or emails.

But if that's the case, what will happen to the WhatsApp messages others send to you when your phone is switched off? Where does it exist? Where is it stored? 

Let's say B's phone is switched off. A sends a WhatsApp message to B. A then switches their phone off. Both phones are now switched off. Does the message exist anywhere?

B switches their phone on now. (A's phone is still switched off). Will B receive the WhatsApp message sent by A?

The answer is yes. And the answer is "servers".

A server is just a computer that is kept on and connected to internet all the time.

When A sends a WhatsApp message to B, A's message is not directly send from A's phone to B's phone. Instead, A's message is send from A's phone to a computer owned by the WhatsApp company. This computer is always kept on. This computer might physically be located in California, or London, or Mumbai. We do not know for sure. But WhatsApp knows. And "server" is just another word for this computer that is always on.

This server sends the message then to B whenever possible. If B is online, it will immediately send that message. If B is switched off and later comes back online, the server will send the message to B then.

That's what a server is. A computer that's always online.

It is not just WhatsApp. Almost everything in today's internet works through servers. If you're reading this through an email, you are probably getting that email off your email providers' server (Gmail/Yahoo/whoever). If you're seeing this on a blog, you connected to Blogger company's server to download this post to your computer.

***

Cloud / Cloud server

Cloud is just a fancy name for servers run by big companies like Amazon/Google/Microsoft. When I run a computer at my home and keep it always online, it is called just a "server". But when a capitalist company runs a computer at their air-conditioned, high security, custom built buildings, it is called a "cloud server", or sometimes simply "cloud".

 ***

We will look at some related words like "client", "database", "website", "protocol", etc in the next post.

Analysis of v-safe response data

 Amar Jesani shared in mfc group link to an article about v-safe data release.

The actual data could be downloaded from this website called icandecide.

The 5GB file can be extracted with p7zip to a 25GB CSV file.

$ md5sum consolidated_health_checkin.zip
53ff7a8153f44eaab4166f722b726fe1  consolidated_health_checkin.zip
$ md5sum consolidated_health_checkin.csv
345cf6ca148832141260aab8638bf0dc  consolidated_health_checkin.csv

$ wc -l consolidated_health_checkin.csv
144856044 consolidated_health_checkin.csv

(That's 144 million records in this CSV file)

$ head -n 5 consolidated_health_checkin.csv
SURVEY_STATIC_ID,REGISTRANT_CODE,RESPONSE_ID,STARTED_ON,STARTED_ON_TIME,DAYS_SINCE,ABDOMINAL_PAIN,CHILLS,DIARRHEA,FATIGUE,FEELING_TODAY,FEVER,HAD_SYMPTOMS,HEADACHE,HEALTH_IMPACT,HEALTH_NOW,HEALTH_NOW_COMPARISON,VACCINE_CAUSED_HEALTH_ISSUES,HEALTHCARE_VISITS,ITCHING,JOINT_PAINS,MUSCLE_OR_BODY_ACHES,NAUSEA,PAIN,PREGNANT,PREGNANCY_TEST,RASH_OUTSIDE_INJECTION_SITE,REDNESS,SITE_REACTION,SWELLING,SYSTEMIC_REACTION,TEMPERATURE_CELSIUS,TEMPERATURE_FAHRENHEIT,TEMPERATURE_READING,TESTED_POSITIVE,TESTED_POSITIVE_DATE,VOMITING,DURATION_MINS,PREFERRED_LANGUAGE
vsafe-0-day-dose1,222-10271-84782,s244305050865137831057660547899056617007,12/31/2020,4:55:13 PM,0,,,,,Good,No,,Mild,N/A,,,,,,,,,Mild,,,,,Pain,,Headache,,,,,,,,English
vsafe-0-day-dose1,222-10325-02776,s258811629454233188277362395339553379505,05/19/2021,3:16:15 PM,0,,,,,Good,No,,,N/A,,,,,,,,,,,,,,None,,None,,,,,,,0.85,English
vsafe-0-day-dose1,222-10368-05218,s256518678527351061889187968276580937945,04/27/2021,4:11:31 PM,0,,,,,Good,No,,,N/A,,,,,,,,,,No,,,,None,,None,,,,,,,0.72,English
vsafe-0-day-dose1,222-10453-23273,s245552707728162053684731534374544736656,01/12/2021,3:31:16 PM,0,,,,,Good,No,,,N/A,,,,,,,,,,No,,,,None,,None,,,,,,,,English


As you can see there are many columns, which we will have to decode.

Combing through the whole file again and again is taking a lot of time on my computer. So I decided to write a python script that'll do all analysis in one pass of the file.

But that was taking even more time.

So I decided to put this data into postgreSQL to do the analysis.


$ sudo -u postgres createuser health

$ sudo -u postgres createdb vsafe -O health

$ cat load.sql
SET datestyle TO dmy;
CREATE table if not exists checkin (
        SURVEY_STATIC_ID varchar, -- eg: vsafe-0-day-dose1
        REGISTRANT_CODE varchar, -- eg: 222-10271-84782
        RESPONSE_ID varchar,
        STARTED_ON DATE,
        STARTED_ON_TIME varchar,
        DAYS_SINCE int,
        ABDOMINAL_PAIN varchar,
        CHILLS varchar,
        DIARRHEA varchar,
        FATIGUE varchar,
        FEELING_TODAY varchar,
        FEVER varchar,
        HAD_SYMPTOMS varchar,
        HEADACHE varchar,
        HEALTH_IMPACT varchar,
        HEALTH_NOW varchar,
        HEALTH_NOW_COMPARISON varchar,
        VACCINE_CAUSED_HEALTH_ISSUES varchar,
        HEALTHCARE_VISITS varchar,
        ITCHING varchar,
        JOINT_PAINS varchar,
        MUSCLE_OR_BODY_ACHES varchar,
        NAUSEA varchar,
        PAIN varchar,
        PREGNANT varchar,
        PREGNANCY_TEST varchar,
        RASH_OUTSIDE_INJECTION_SITE varchar,
        REDNESS varchar,
        SITE_REACTION varchar,
        SWELLING varchar,
        SYSTEMIC_REACTION varchar,
        TEMPERATURE_CELSIUS varchar,
        TEMPERATURE_FAHRENHEIT varchar,
        TEMPERATURE_READING  varchar,
        TESTED_POSITIVE varchar,
        TESTED_POSITIVE_DATE varchar,
        VOMITING varchar,
        DURATION_MINS FLOAT,
        PREFERRED_LANGUAGE varchar
);

\COPY checkin FROM 'consolidated_health_checkin.csv' DELIMITER ',' CSV HEADER

$ psql -U health vsafe -f load.sql
...
COPY 144856043
 

That took about 25G space as well.

Beautiful. Now we can do all kinds of querying.


Actually, not yet. There's one more thing we have to do. Create some indexes for making queries easier.


CREATE INDEX checkin_health_impact_idx ON public.checkin USING btree (health_impact);



Now, there are some issues with this data. For example:

ERROR: could not create unique index "checkin_pk"
  Detail: Key (response_id)=(s252082802016465320050574992159464366472) is duplicated.
  Where: parallel worker


response_id is duplicated, although it looks like every response might be unique.


But let's ignore that now for an interesting query result:


 

That's the distribution of the Health Impact column. 81 million responses say N/A, 56 million responses include no value (null) for this column  and the tail kind of begins there.

When I do select count(distinct(registrant_code)) from checkin; I get 9,552,127 which means only 9.5 million registrant_codes are included in the dataset. Since v-safe allows adults to respond on behalf of children, it is probably likely that there are more individuals in the dataset than the registrant_codes.

Then I ran select count(distinct(registrant_code)) from checkin where health_impact like '%Get care from a doctor or other healthcare professional%'; and it returned 797,396. Which means at least 797K people checked this option (with or without other options)


Now let us look at the variable of interest, healthcare_visits. The query I ran is select healthcare_visits, count(*) from checkin where health_impact like '%Get care from a doctor or other healthcare professional%' group by healthcare_visits ;

The result is

 

Note that I haven't deduplicated by registrant_code here. 

So I tried a different query: select count(*) from checkin where healthcare_visits  like '%Hospitalization%'; the answer to which is 83,690.

Let us try deduplicating by registrant_code on that:

select count(distinct(registrant_code)) from checkin where healthcare_visits  like '%Hospitalization%'; returns: 71,911

Which means, there's some amount of duplication in the row data as to registrant_codes and reports. In other words, from the same registrant_code, you can have multiple reports of Hospitalization.

This data is rather messy and I'm not exactly sure how icandecide is arriving at "individual" in their numbers because all I see are registrant_code.

Now, on to some more interesting stuff. What is the distribution of systemic_reaction in registrant_codes who reported Hospitalization?

select systemic_reaction, count(distinct(registrant_code))from checkin where healthcare_visits  like '%Hospitalization%' group by systemic_reaction ;

That turned out disappointing because the result was 68,170 NULL fields.

 

 But among the non-null fields, "None", "Fatigue or tiredness", "Headache", etc are leading. (Do note that this is a multi-value column and there could theoretically be a symptom that appears in the tail of this column multiple times thus occurring more number of times than these ones.)

I also looked at the other files available for download.

It seems like the Consolidated_health_checkin_u3[1].zip must be under 3 children. The consolidated_registrants[1].zip file makes me think that each registrant_code actually uniquely identifies an individual. Because children are having separate registrant_code with guardian registrant_code mapped in this file. The other files are about race/ethnicity and vaccine that was administered.

 

The under 3 file includes 116394 reports. Some of the discrepancies in number between my analysis and ICAN's dashboard probably comes from them adding both these together. 

Don't Jump On Private Healthcare

Follow me

@asdofindia on Twitter
@learnlearnin on Telegram

About Me

My photo
I am a general practitioner rooted in the principles of primary healthcare. I am also a deep generalist and hold many other interests. If you want a medical consultation, please book an appointment When I'm not seeing patients, I code software, advise health-tech startups, and write blogs. Follow me by subscribing to my writings