ngrams
Janis Corona
4/220/2020
This uses a previous python3 script inside R with reticulate to communicate python code within R. This will classify a wellness or health category based on it being one of either articles pulled from the internet on: physical therapy benefits, massage therapy benefits, chiropractic benefits, massage gun benefits, benefits of mental health services, cold stone therapy benefits, or cupping benefits. Then some ‘non professional’ google and twitter massage posts were added in to this selection to see how well the classifier could discriminate between one of the seven professional healthcare recommendations and a non-professional recommendation. This script also uses added data on medical professional or ER visit recommended based on user input. This data now has 9 categories to classify based on user input. Also, the risks, effects, and contraindications were scrubbed from the document of each class if it existed and placed into their corresponding feature of ‘contraindication’ or ‘risksAdverseEffect.’ This should improve the accuracy in classifying a recommendation for user inputs based on his or her requests.
The python modules were sklearn, matplotlib, pandas, numpy, nltk, textBlob, and regex. Some versions that work are later modules, for instance the re package was used that made regex obsolete because it is a build version that replaced regex for my version of python, 3.6.
library(reticulate)## Warning: package 'reticulate' was built under R version 3.6.3conda_list(conda = "auto") ## name python
## 1 Anaconda2 C:\\Users\\m\\Anaconda2\\python.exe
## 2 djangoenv C:\\Users\\m\\Anaconda2\\envs\\djangoenv\\python.exe
## 3 python36 C:\\Users\\m\\Anaconda2\\envs\\python36\\python.exe
## 4 python37 C:\\Users\\m\\Anaconda2\\envs\\python37\\python.exe
## 5 r-reticulate C:\\Users\\m\\Anaconda2\\envs\\r-reticulate\\python.exeWithout having my python IDE, Anaconda, open in the console I want to use the python36 environment, all the environments in Anaconda for python are listed above.
use_condaenv(condaenv = "python36")import pandas as pd
import matplotlib.pyplot as plt
from textblob import TextBlob
import sklearn
import numpy as np
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
import re
import string
import nltk
np.random.seed(47) set.seed(47)The following data table will not show in your Rstudio environment, but python inside your python IDE will store the table.
modalities = pd.read_csv('benefitsContraindications3.csv', encoding = 'unicode_escape') print(modalities.head())## Document ... risksAdverseEffects
## 0 Chiropractic adjustments and treatments serve ... ... NaN
## 1 \r\nUnitedHealthcare Combats Opioid Crisis wit... ... NaN
## 2 The Safety of Chiropractic Adjustments\r\nBy ... ... Risks and side effects associated with chiropr...
## 3 Advanced Chiropractic Relief: 8 Key Benefits o... ... NaN
## 4 Heading to the spa can be a pampering treat, b... ... NaN
##
## [5 rows x 6 columns]print(modalities.tail())## Document ... risksAdverseEffects
## 82 General guidelines - When to visit an emergenc... ... NaN
## 83 \r\nHow to know where to go for sudden health ... ... NaN
## 84 How to Know If You Need to Go to the E.R. With... ... NaN
## 85 When to call 911 or go to an emergency room im... ... NaN
## 86 Is It an Emergency?\r\n\r\nConditions we treat... ... NaN
##
## [5 rows x 6 columns]print(modalities.shape)## (87, 6)print(modalities.columns)## Index(['Document', 'Source', 'Topic', 'InternetSearch', 'Contraindications',
## 'risksAdverseEffects'],
## dtype='object')import regex
def preprocessor(text):
text = regex.sub('<[^>]*>', '', text)
emoticons = regex.findall('(?::|;|=)(?:-)?(?:\)|\(|D|P)', text)
text = regex.sub('[\W]+', ' ', text.lower()) +\
' '.join(emoticons).replace('-', '')
return textmodalities.tail()## Document ... risksAdverseEffects
## 82 General guidelines - When to visit an emergenc... ... NaN
## 83 \r\nHow to know where to go for sudden health ... ... NaN
## 84 How to Know If You Need to Go to the E.R. With... ... NaN
## 85 When to call 911 or go to an emergency room im... ... NaN
## 86 Is It an Emergency?\r\n\r\nConditions we treat... ... NaN
##
## [5 rows x 6 columns]Reorder the observations so that they are mixed and not grouped together as they are in the original file.
import numpy as np
modalities = modalities.reindex(np.random.permutation(modalities.index))
print(modalities.head())## Document ... risksAdverseEffects
## 66 Do you prefer Pastel Pink vibes or All Black v... ... NaN
## 40 \r\nWhat Is Cupping Therapy?And Should You Try... ... NaN
## 14 What Are the Health Benefits of Massage?\r\n\r... ... NaN
## 46 ll You Need To Know About Massage Gun\r\n31 Au... ... NaN
## 47 \r\nBenefits of Vibration and Percussion Thera... ... NaN
##
## [5 rows x 6 columns]print(modalities.tail())## Document ... risksAdverseEffects
## 72 I adore men who massage me all over my body ... NaN
## 8 BENEFITS OF MASSAGE\r\n\r\nYou know that post-... ... NaN
## 71 Hey twitter I'm looking for a lady to massage ... ... NaN
## 6 25 Reasons to Get a Massage\r\n\r\nNovember 5,... ... NaN
## 7 7 Benefits of Massage Therapy\r\n\r\nMassage t... ... NaN
##
## [5 rows x 6 columns]modalities.columns## Index(['Document', 'Source', 'Topic', 'InternetSearch', 'Contraindications',
## 'risksAdverseEffects'],
## dtype='object')modalities.groupby('Topic').describe()## Document ... risksAdverseEffects
## count ... freq
## Topic ...
## ER 6 ... NaN
## Not Professional 16 ... NaN
## chiropractic benefits 8 ... 1
## cold stone benefits 10 ... NaN
## cupping benefits 10 ... 1
## massage benefits 12 ... NaN
## massage gun benefits 10 ... NaN
## mental health services benefits 6 ... NaN
## physical therapy benefits 9 ... NaN
##
## [9 rows x 20 columns]modalities['length'] = modalities['Document'].map(lambda text: len(text))
print(modalities.head())## Document ... length
## 66 Do you prefer Pastel Pink vibes or All Black v... ... 133
## 40 \r\nWhat Is Cupping Therapy?And Should You Try... ... 5471
## 14 What Are the Health Benefits of Massage?\r\n\r... ... 1820
## 46 ll You Need To Know About Massage Gun\r\n31 Au... ... 5521
## 47 \r\nBenefits of Vibration and Percussion Thera... ... 3362
##
## [5 rows x 7 columns]modalities.length.plot(bins=20, kind='hist')
plt.show()
modalities.length.describe()## count 87.000000
## mean 3842.275862
## std 3103.734864
## min 27.000000
## 25% 1366.500000
## 50% 3191.000000
## 75% 5936.500000
## max 12846.000000
## Name: length, dtype: float64print(list(modalities.Document[modalities.length > 3800].index))## [40, 46, 44, 49, 30, 17, 5, 25, 83, 58, 29, 1, 38, 54, 16, 2, 28, 26, 3, 42, 21, 27, 22, 9, 35, 34, 53, 41, 50, 55, 45, 84, 59, 23, 51, 7]print(list(modalities.Topic[modalities.length > 3800]))## ['cupping benefits', 'massage gun benefits', 'cupping benefits', 'massage gun benefits', 'mental health services benefits', 'physical therapy benefits', 'massage benefits', 'mental health services benefits', 'ER', 'cold stone benefits', 'mental health services benefits', 'chiropractic benefits', 'cupping benefits', 'massage gun benefits', 'physical therapy benefits', 'chiropractic benefits', 'mental health services benefits', 'mental health services benefits', 'chiropractic benefits', 'cupping benefits', 'physical therapy benefits', 'mental health services benefits', 'physical therapy benefits', 'massage benefits', 'cupping benefits', 'chiropractic benefits', 'massage gun benefits', 'cupping benefits', 'massage gun benefits', 'cold stone benefits', 'massage gun benefits', 'ER', 'cold stone benefits', 'physical therapy benefits', 'massage gun benefits', 'massage benefits']modalities.hist(column='length', by='Topic', bins=5)
plt.show()
def split_into_tokens(review):
return TextBlob(review).wordsmodalities.Document.head().apply(split_into_tokens)## 66 [Do, you, prefer, Pastel, Pink, vibes, or, All...
## 40 [What, Is, Cupping, Therapy, And, Should, You,...
## 14 [What, Are, the, Health, Benefits, of, Massage...
## 46 [ll, You, Need, To, Know, About, Massage, Gun,...
## 47 [Benefits, of, Vibration, and, Percussion, The...
## Name: Document, dtype: objectTextBlob("hello world, how is it going?").tags # list of (word, POS) pairs## [('hello', 'JJ'), ('world', 'NN'), ('how', 'WRB'), ('is', 'VBZ'), ('it', 'PRP'), ('going', 'VBG')]import nltk
nltk.download('stopwords')## True
##
## [nltk_data] Downloading package stopwords to
## [nltk_data] C:\Users\m\AppData\Roaming\nltk_data...
## [nltk_data] Package stopwords is already up-to-date!from nltk.corpus import stopwords
stop = stopwords.words('english')
stop = stop + [u'a',u'b',u'c',u'd',u'e',u'f',u'g',u'h',u'i',u'j',u'k',u'l',u'm',u'n',u'o',u'p',u'q',u'r',u's',u't',u'v',u'w',u'x',u'y',u'z']
def split_into_lemmas(review):
#review = unicode(review, 'iso-8859-1')
review = review.lower()
#review = unicode(review, 'utf8').lower()
#review = str(review).lower()
words = TextBlob(review).words
# for each word, take its "base form" = lemma
return [word.lemma for word in words if word not in stop]
modalities.Document.head().apply(split_into_lemmas)## 66 [prefer, pastel, pink, vibe, black, vibe, lil,...
## 40 [cupping, therapy, try, 're, definitely, going...
## 14 [health, benefit, massage, many, type, massage...
## 46 [need, know, massage, gun, 31, august, 2019, 3...
## 47 [benefit, vibration, percussion, therapy, vibr...
## Name: Document, dtype: objectbow_transformerNgrams = CountVectorizer(analyzer=split_into_lemmas,ngram_range=(2,2)).fit(modalities['Document'])
print(len(bow_transformerNgrams.vocabulary_))
## 5102modality4 = modalities['Document'][40]
print(modality4)##
## What Is Cupping Therapy?And Should You Try It?
##
## You're definitely going to have bruises, JSYK.
## By Annie Daly
## Jun 26, 2018
## Getty Images
##
## Cupping is the wellness trend that just refuses to die.
##
## Seriously?raise your hand if you thought cupping therapy would die down back in 2016 after Michael Phelps permanently exited the pool (it me).
##
## No such luck: Two years later and celebs are still participating in the cupping trend (I see you, Kaley Cuoco and Busy Phillips).
##
## But uh, what exactly is cupping?and what is it used for?
## Alright, WTF is cupping?
##
## Cupping is an ancient Chinese therapy that?s based on the belief that certain health problems can be caused by stagnant blood and a poor energy flow through your body.
##
## To fix or prevent those health issues, cupping practitioners apply cups?typically glass or silicone?to your skin to create a pressure that sucks your skin inward, according to the National Center for Complementary and Integrative Medicine.
## Advertisement - Continue Reading Below
##
## Cupping practitioners usually place cups on a person's back?though face cupping is becoming *a thing* now too?where they'll either leave the cups in place, or slide them around using lotion or oil, per the NCCIM.
##
## Either way, the pressure that the cups create draws blood to the affected area, increasing your blood flow overall, says Chiti Parikh, M.D., integrative medicine practitioner at the Weill Cornell Medicine Integrative Health and Wellbeing Center in New York City.
##
## ?That increased blood flow can relieve muscle tension, improve circulation, and reduce inflammation," says Parikh.
## What's cupping even used for?
##
## Generally, cupping is used to treat chronic pain?back pain and headaches, in particular, says Parikh. ?It?s all about getting rid of musculoskeletal pain, which is often a physical manifestation of chronic stress,? he adds.
##
## More often than not, that chronic stress manifests in how you carry yourself. ?We?re often tensing our muscles when we?re stressed?especially when we?re hunched over our computers and our phones?and that muscle tension can result in physical pain, which is what cupping helps reduce," says Parikh.
## Related Story
## What's Causing Your Chronic Back Pain?
##
## Many people also claim that cupping can help with detoxification, but because "detoxing" really isn't a thing, what they mean is that they are relieving inflammation in the area, Parikh clarifies.
##
## ?When people are suffering from physical pain, that means that inflammation has increased locally in that area. Cupping, then, improves blood circulation in that area by attracting immune cells to that location to increase the repair and recovery process, so that the swelling can go down,? explains Parikh.
##
## While there are studies that claim cupping therapy can help reduce chronic pain, like a 2016 study published in the journal Evidence Based Complementary Alternative Medicine, the research is still inconclusive.
##
## That results of that study in particular, as well as many other studies regarding cupping, may also be due to the placebo effect (i.e., simply believing something is working).
## So, can cupping reduce my stress?
##
## Cupping may be helpful in treating the more physical manifestations of stress. But, as we all know, physical symptoms are only part of the full story.
##
## That's why cupping works best when paired with acupuncture to help the mental side of stress, says Parikh. ?Acupuncture releases endorphins, and may be better able to help manage the root cause of your mental stress and anxiety in a more holistic way," she explains. "That?s why the combo of cupping and acupuncture works really synergistically to manage your stress from all areas."
## Related Story
## The Best Way To De-Stress For Your Zodiac Sign
##
## Plus, while trying cupping on its own may help treat physical pain in more acute conditions, the best and most long-lasting relief comes when you go for the combo. ?The effect is simply more sustainable that way,? she concludes. ?Cupping is hardly ever recommended on its own.?
## Should I try cupping or nah?
##
## In short: It's probably not going to do any damage?and hey, it might even help that back pain that's been bugging you for weeks, so go ahead and give it a try.
## Advertisement - Continue Reading Below
##
## Something to keep in mind: Those cups are definitely going to leave big, hickey-like bruises on your skin that can last anywhere from a few days to two weeks?so if that's not something you're interested in, maybe steer clear.
##
## (It should be noted, however, that the color of the mark varies from person to person and doesn't signify how well the cupping worked.)
##
## You'll also want to steer clear of cupping if you?re on blood thinners, have trouble with bleeding or clotting, or you have an open wound, cautions Parikh. The same goes for those with very sensitive or thin skin, she adds. ?You should also avoid cupping on any areas where you have delicate skin, because it can cause tearing."
##
## If these restrictions do not apply to you and you want to give cupping a whirl, it?s important to find a good practitioner. Parikh says the best cupping practitioners are actually?you guessed it?acupuncturists.
##
## ?To find a good one in your area, I would recommend asking an integrative medicine doctor for a recommendation," says Parikh. "If you don?t know one, your regular doctor will usually be able to refer you to a good one."
##
## Now, go forth and let the cupping beginbow4 = bow_transformerNgrams.transform([modality4])
print(bow4)## (0, 3) 2
## (0, 5) 2
## (0, 7) 7
## (0, 75) 2
## (0, 78) 1
## (0, 94) 1
## (0, 184) 2
## (0, 209) 1
## (0, 242) 1
## (0, 245) 3
## (0, 247) 1
## (0, 248) 1
## (0, 251) 2
## (0, 290) 2
## (0, 298) 1
## (0, 319) 1
## (0, 354) 1
## (0, 355) 4
## (0, 361) 1
## (0, 384) 1
## (0, 400) 1
## (0, 417) 1
## (0, 423) 1
## (0, 437) 2
## (0, 451) 7
## : :
## (0, 4721) 1
## (0, 4735) 2
## (0, 4740) 1
## (0, 4744) 1
## (0, 4824) 3
## (0, 4827) 1
## (0, 4828) 2
## (0, 4847) 1
## (0, 4914) 2
## (0, 4935) 4
## (0, 4949) 2
## (0, 4955) 1
## (0, 4958) 2
## (0, 4962) 1
## (0, 4963) 1
## (0, 4985) 1
## (0, 5013) 2
## (0, 5014) 1
## (0, 5016) 1
## (0, 5028) 2
## (0, 5029) 1
## (0, 5038) 1
## (0, 5047) 1
## (0, 5058) 1
## (0, 5070) 1modalities_bow = bow_transformerNgrams.transform(modalities['Document'])
print('sparse matrix shape:', modalities_bow.shape)## sparse matrix shape: (87, 5102)print('number of non-zeros:', modalities_bow.nnz)## number of non-zeros: 17813print('sparsity: %.2f%%' % (100.0 * modalities_bow.nnz / (modalities_bow.shape[0] * modalities_bow.shape[1])))## sparsity: 4.01%modalities_bow## <87x5102 sparse matrix of type '<class 'numpy.int64'>'
## with 17813 stored elements in Compressed Sparse Row format>
# Split/splice into training ~ 80% and testing ~ 20%
modalities_bow_train = modalities_bow[:69]
modalities_bow_test = modalities_bow[69:]
modalities_sentiment_train = modalities['Topic'][:69]
modalities_sentiment_test = modalities['Topic'][69:]
print(modalities_bow_train.shape)## (69, 5102)print(modalities_bow_test.shape)## (18, 5102)print## <built-in function print>modalities_sentiment = MultinomialNB().fit(modalities_bow_train, modalities_sentiment_train)print('predicted:', modalities_sentiment.predict(bow4)[0])## predicted: cupping benefitsprint('expected:', modalities.Topic[40])## expected: cupping benefitspredictions = modalities_sentiment.predict(modalities_bow_test)
#print(predictions)
prd = pd.DataFrame(predictions)
prd.columns=['predictions']
prd.index=modalities_sentiment_test.index
pred=pd.concat([pd.DataFrame(prd),modalities_sentiment_test],axis=1)
print(pred)## predictions Topic
## 82 ER ER
## 41 cupping benefits cupping benefits
## 52 massage gun benefits massage gun benefits
## 65 massage benefits Not Professional
## 50 massage gun benefits massage gun benefits
## 55 massage benefits cold stone benefits
## 45 massage gun benefits massage gun benefits
## 84 ER ER
## 86 ER ER
## 59 cold stone benefits cold stone benefits
## 23 physical therapy benefits physical therapy benefits
## 80 massage gun benefits Not Professional
## 51 massage gun benefits massage gun benefits
## 72 massage gun benefits Not Professional
## 8 massage benefits massage benefits
## 71 massage benefits Not Professional
## 6 massage benefits massage benefits
## 7 massage benefits massage benefitsprint('accuracy', accuracy_score(modalities_sentiment_test, predictions))## accuracy 0.7222222222222222print('confusion matrix\n', confusion_matrix(modalities_sentiment_test, predictions))## confusion matrix
## [[3 0 0 0 0 0 0]
## [0 0 0 0 2 2 0]
## [0 0 1 0 1 0 0]
## [0 0 0 1 0 0 0]
## [0 0 0 0 3 0 0]
## [0 0 0 0 0 4 0]
## [0 0 0 0 0 0 1]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(modalities_sentiment_test, predictions))## precision recall f1-score support
##
## ER 1.00 1.00 1.00 3
## Not Professional 0.00 0.00 0.00 4
## cold stone benefits 1.00 0.50 0.67 2
## cupping benefits 1.00 1.00 1.00 1
## massage benefits 0.50 1.00 0.67 3
## massage gun benefits 0.67 1.00 0.80 4
## physical therapy benefits 1.00 1.00 1.00 1
##
## accuracy 0.72 18
## macro avg 0.74 0.79 0.73 18
## weighted avg 0.62 0.72 0.64 18
##
##
## C:\Users\m\Anaconda2\envs\python36\lib\site-packages\sklearn\metrics\classification.py:1437: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.
## 'precision', 'predicted', average, warn_for)From the above, precision accounts for type 1 errors (how many real negatives classified as positives-False Positives: TP/(TP+FP)) and type 2 errors (how many real posiives classified as negatives-False Negatives: TP/(TP+FN)) are part of recall. We can see that in our testing set of samples, by looking at precision, the ER, cold stone benefits, cupping benefits, and physical therapy benefits classes were correctly classified when classified at all. But for recall, all except for cold stone benefits and the Not Professional class were mis-classified as cold stone benefits when the class was not a cold stone benefit. Also, for precision, the massage gun and massage benefits as well as the Not Professional classes were mis-classified as each such class incorrectly. So, when the classifier predicted the class as being massage gun benefits the classification was correct 67% of the time, and half the time for massage benefits as far as precision is concerned. And when the classifications of ER, cupping, massage, massage gun, and physical therapy benefits were classifed, they were all ‘found’ or recalled, there were no misclassifications of those classes.
A note with precision and recall, because when learning from different sources, they have differnt nmemonics for remembering, even with the way they set up their Type I and Type II diagram, and that could leave to confusion. But it is important to make it simple and not complex nor introduce more confusion when not needed. Recall is analogous to saying, ‘out of all those that were class A, we classified them with some percent accuracy.’ For precision, it is analogous to saying, ‘out of all the classifications we made for class A, we were right this percent of the time.’ Or find what works for you. Precision is Type 1 error and recall is a Type 2 error, some nmemonics say to use how many negatives? One or two. If predicted positive when negative, its a false negative or type 2 error, and if predicted negative when positive, it is a false positive or type 1 error.
Aside: If your concern is with improving recall or improving precision, you would need to tune your predictive model to get better prediction or recall and put aside the need for the accuracy to be close to 100%. Because if accuracy is 98% and your recall is 60%, that is not going to help you, if say you needed to find all of the tumor cells and only found 60% of them. Same, for precision, if your accuracy is 98% and your precision is 80% and recall 90%, then if your predicting the next day of a stock price increasing and misclassify it as such when it decreases, your precision needs to be improved. Because you may have classified 90% of those days that increased, but you misclassified some decreasing days by 80%, and could have bought that stock. When in doubt, do as the nerds do and set up 10 analogous examples of real-life outcomes and see if its true, or you could stand in a corner until you fall asleep and dream about what is a type 1 and type 2. Or depending on your access to google just google it. But when classifying, you should definitely make sure you have a clear understanding of the difference and necessity for recall and precision. If this was a movie, there would be an unanswered question as to how to tune for these improvements in precision or recall. Since, the above uses multinomial naive bayes, the probabilities are based on the principle of garbage in and garbage out. You would have to get better data, more relevant features, exclude features, play with the testing and training sets, remove outliers, or normalize the data to include outliers by taking the log, and getting to know the sometimes 100s of features (or appearance of) your algorithm if using a package like nltk on tokenizing words. You will see this later in the list of nltk attributes in the other algorithm and model testing on this dataset.
modalitiesu = modalities.Topic.unique()
mus = np.sort(modalitiesu)
mus## array(['ER', 'Not Professional', 'chiropractic benefits',
## 'cold stone benefits', 'cupping benefits', 'massage benefits',
## 'massage gun benefits', 'mental health services benefits',
## 'physical therapy benefits'], dtype=object)
def predict_modality(new_review):
new_sample = bow_transformerNgrams.transform([new_review])
pr = np.around(modalities_sentiment.predict_proba(new_sample),2)
print(new_review,'\n\n', pr)
print('\n\nThe respective order:\n 0-ER\n 1-Non Professional\n 2-chiropractic therapy\n 3-cold stone therapy\n 4-cupping therapy\n 5-massage therapy\n 6-massage gun therapy\n 7-mental health therapy\n 8-physical therapy\n\n')
if (pr[0][0] == max(pr[0])):
print('The max probability is Emergency Room services for this recommendation with ', pr[0][0]*100,'%')
elif (pr[0][1] == max(pr[0])):
print('The max probability is Non-Professional services for this recommendation with ', pr[0][1]*100,'%')
elif (pr[0][2] == max(pr[0])):
print('The max probability is chiropractic therapy for this recommendation with ', pr[0][2]*100,'%')
elif (pr[0][3] == max(pr[0])):
print('The max probability is cold stone massage for this recommendation with ', pr[0][3]*100,'%')
elif (pr[0][4] == max(pr[0])):
print('The max probability is cupping therapy for this recommendation with ', pr[0][4]*100,'%')
elif (pr[0][5] == max(pr[0])):
print('The max probability is massage therapy for this recommendation with ', pr[0][5]*100,'%')
elif (pr[0][6] == max(pr[0])):
print('The max probability is massage gun therapy for this recommendation with ', pr[0][6]*100,'%')
elif (pr[0][7] == max(pr[0])):
print('The max probability is mental health therapy for this recommendation with ', pr[0][7]*100,'%')
else:
print('The max probability is physical therapy for this recommendation with ', pr[0][8]*100,'%')
print('-----------------------------------------\n\n')predict_modality('Headaches, body sweats, depressed.')## Headaches, body sweats, depressed.
##
## [[0. 0.03 0.33 0.07 0.02 0.51 0.01 0.01 0.01]]
##
##
## The respective order:
## 0-ER
## 1-Non Professional
## 2-chiropractic therapy
## 3-cold stone therapy
## 4-cupping therapy
## 5-massage therapy
## 6-massage gun therapy
## 7-mental health therapy
## 8-physical therapy
##
##
## The max probability is massage therapy for this recommendation with 51.0 %
## -----------------------------------------predict_modality('sleepless, energy depraved, cold, tension')## sleepless, energy depraved, cold, tension
##
## [[0. 0. 0.01 0.94 0.04 0.01 0. 0. 0. ]]
##
##
## The respective order:
## 0-ER
## 1-Non Professional
## 2-chiropractic therapy
## 3-cold stone therapy
## 4-cupping therapy
## 5-massage therapy
## 6-massage gun therapy
## 7-mental health therapy
## 8-physical therapy
##
##
## The max probability is cold stone massage for this recommendation with 94.0 %
## -----------------------------------------predict_modality('body aches from working out')## body aches from working out
##
## [[0. 0.12 0.05 0.16 0.11 0.22 0.19 0.07 0.08]]
##
##
## The respective order:
## 0-ER
## 1-Non Professional
## 2-chiropractic therapy
## 3-cold stone therapy
## 4-cupping therapy
## 5-massage therapy
## 6-massage gun therapy
## 7-mental health therapy
## 8-physical therapy
##
##
## The max probability is massage therapy for this recommendation with 22.0 %
## -----------------------------------------predict_modality('can\'t move my arm. stuck at home. worried about my neck.')## can't move my arm. stuck at home. worried about my neck.
##
## [[0.03 0. 0.2 0.02 0.04 0.37 0.13 0. 0.2 ]]
##
##
## The respective order:
## 0-ER
## 1-Non Professional
## 2-chiropractic therapy
## 3-cold stone therapy
## 4-cupping therapy
## 5-massage therapy
## 6-massage gun therapy
## 7-mental health therapy
## 8-physical therapy
##
##
## The max probability is massage therapy for this recommendation with 37.0 %
## -----------------------------------------predict_modality('breathing ragged, tired, headaches, dizzy, nausious ')## breathing ragged, tired, headaches, dizzy, nausious
##
## [[0.12 0.03 0.36 0.06 0.02 0.37 0.01 0.01 0.01]]
##
##
## The respective order:
## 0-ER
## 1-Non Professional
## 2-chiropractic therapy
## 3-cold stone therapy
## 4-cupping therapy
## 5-massage therapy
## 6-massage gun therapy
## 7-mental health therapy
## 8-physical therapy
##
##
## The max probability is massage therapy for this recommendation with 37.0 %
## -----------------------------------------predict_modality("relief from this pain. can't sleep. feet hurt. chills.")## relief from this pain. can't sleep. feet hurt. chills.
##
## [[0. 0. 0.3 0. 0.04 0.56 0.01 0. 0.09]]
##
##
## The respective order:
## 0-ER
## 1-Non Professional
## 2-chiropractic therapy
## 3-cold stone therapy
## 4-cupping therapy
## 5-massage therapy
## 6-massage gun therapy
## 7-mental health therapy
## 8-physical therapy
##
##
## The max probability is massage therapy for this recommendation with 56.00000000000001 %
## -----------------------------------------predict_modality('love this place better than others')## love this place better than others
##
## [[0.01 0.1 0.09 0.05 0.3 0.02 0.01 0.39 0.03]]
##
##
## The respective order:
## 0-ER
## 1-Non Professional
## 2-chiropractic therapy
## 3-cold stone therapy
## 4-cupping therapy
## 5-massage therapy
## 6-massage gun therapy
## 7-mental health therapy
## 8-physical therapy
##
##
## The max probability is mental health therapy for this recommendation with 39.0 %
## -----------------------------------------library(reticulate)conda_list(conda = "auto") ## name python
## 1 Anaconda2 C:\\Users\\m\\Anaconda2\\python.exe
## 2 djangoenv C:\\Users\\m\\Anaconda2\\envs\\djangoenv\\python.exe
## 3 python36 C:\\Users\\m\\Anaconda2\\envs\\python36\\python.exe
## 4 python37 C:\\Users\\m\\Anaconda2\\envs\\python37\\python.exe
## 5 r-reticulate C:\\Users\\m\\Anaconda2\\envs\\r-reticulate\\python.exeWithout having my python IDE, Anaconda, open in the console I want to use the python36 environment, all the environments in Anaconda for python are listed above.
use_condaenv(condaenv = "python36")import pandas as pd
import matplotlib.pyplot as plt
from textblob import TextBlob
import sklearn
import numpy as np
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
import re
import string
import nltk
np.random.seed(45678) The first part of the following code uses the Random Forest Classifier (RFC) and the Gradient Boosting Classifier (GBC) to categorize the recommendation based on three separate types of document term matrix (dtm) tokenizations, Count Vectorizer, Term Frequency-Inverse Document Frequency (TF-IDF) Vectorizer, and N-grams Vectorizer. This first part also uses Lemmatization to get the more ideal word meaning root word. And the training model uses 80% of the samples and 20% to test the model on within each type of vectorized and lemmatized token.
The second part of the following will keep all the same but change the training set to 85% and the testing set to 15% while still using lemmatization.
The third part will keep the first part the same but only change the lemmatization to stemmed word roots.
The fourt part will keep the same third part settings but change the testing set to 15% and the training set to 85%.
Those four sections of variations will allow us to contrast and compare which setting worked best for recall, precision, and accuracy within each vectorized and stemmed/lemmatized tokens of the dtms for either RFC or GBC.
First part: Lemmatized Tokens & 80/20 Train/Test split & RFC | GBC
Count Vectorizer RFC and GBC
stopwords = nltk.corpus.stopwords.words('english')
ps=nltk.PorterStemmer()
wn=nltk.WordNetLemmatizer()
data=pd.read_csv("benefitsContraindications3.csv", encoding='unicode_escape')
data.columns=['Document','Source','Topic','InternetSearch','Contraindications','RisksSideEffects']data <- read.csv('benefitsContraindications3.csv',sep=',',header=TRUE, na.strings=c('',' ','NA'))
colnames(data)## [1] "Document" "Source" "Topic"
## [4] "InternetSearch" "Contraindications" "risksAdverseEffects"head(data,5)## Document
## 1 Chiropractic adjustments and treatments serve the needs of millions of people around the world.\n\n \n\nAdjustments offer effective, non-invasive and cost-effective solutions to neck and back pain, as well as a myriad of other medical issues.\n\nHave you ever stopped to wonder how many of us suffer from neck and back stiffness or pain?\n\nApart from the obvious discomfort, simple daily tasks such as driving a car, crossing a busy street and picking things up from the floor can become all too challenging for individuals experiencing such pain.\n\nAs anyone who has experienced pain would know, having restricted movement can be debilitating and unfortunately, our busy world doesn?t allow for us to stop.\n\n \nSome of the benefits of long-term chiropractic care include:\n\n Chiropractors can identify mechanical issues that cause spine-related pain and offer a series of adjustments that provide near immediate relief. Following appointments, patients often report feeling their symptoms noticeably better.\n When a chiropractor performs an adjustment, they can help restore movement in joints that have ?locked up?. This becomes possible as treatment allows muscles surrounding joints to relax, thereby reducing joint stiffness.\n Many factors affect health, including exercise patterns, nutrition, sleep, heredity and the environment in which we live. Rather than just treat symptoms of the disease, chiropractic care focuses on a holistic approach to naturally maintain health and resist disease.\n Chiropractic adjustments help restore normal function and movement to the entire body. Many patients report an improvement in their ability to move with efficiency and strength.\n Many patients find delight in the results chiropractic adjustments have on old and chronic injuries. Whether an injury is, in fact, new or old, chiropractic care can help reduce pain, restore mobility and provide quick pain relief to all joints in the body. Such care can help maintain better overall health and thus faster recovery time.\n\nHave you ever noticed that when you are in pain and unable to perform regular or favorite activities, it can put a strain on emotional and mental well-being?\n\nFor example, the increased stress from not being able to properly perform a paid job. This, in turn, can have a negative impact on physical health with increases in heart rate and blood pressure. The domino effect often continues with sleep becoming disturbed, with resulting lethargy and tiredness during the day. Does anyone really feel up to exercising in this state?\n\nChiropractic care is a natural method of healing the body?s communication system and never relies on the use of pharmaceutical drugs or invasive surgery.\n\nDoctors of Chiropractic work collaboratively with other healthcare professionals. Should your condition require the attention of another healthcare profession, that recommendation or referral will be made.\n\n
## 2 \nUnitedHealthcare Combats Opioid Crisis with Non-Opioid Benefits\nPhysical therapy and chiropractic care can prevent or reduce expensive, invasive spinal procedures, such as imaging or surgery, to reduce opioid use and cut costs.\nUnitedHealthcare, opioid, physical therapy, healthcare spending\n\nSource: Thinkstock\nShare on Twitter\n\nBy Kelsey Waddill\n\nOctober 29, 2019 - UnitedHealthcare (UHC) is combatting the opioid epidemic and high healthcare costs with new physical therapy and chiropractic care benefits to prevent, delay, or in some cases substitute for invasive spinal procedures.\n\n?With millions of Americans experiencing low back pain currently or at some point during their lifetimes, we believe this benefit design will help make a meaningful difference by improving health outcomes while reducing costs,? said Anne Docimo, MD, UnitedHealthcare chief medical officer.\n\nLower back pain is in part responsible for sustaining the opioid epidemic and also increases healthcare costs.\nDig Deeper\n\n How Major Payers Provide Substance Abuse Care for Opioid Misuse\n Opioid Overdoses Fall by 2% from 2017 to 2018, CDC Reports\n Physical Therapy, Chiropractic Back Care Cut Opioid Use, Costs\n\nAlthough opioid overdoses fell by two percent from 2017 to 2018 and a legal battles aim to hold pharmaceutical companies accountable, there is no end in sight for the opioid epidemic. Industry professionals are still grappling with the balance between cutting opioid prescriptions will working to reduce patient pain.\n\nCommon conditions such as low back pain bolster the epidemic?s presence, with clinicians still prescribing the opioids against best practice recommendations. According to a recent OptumLabs study, 9 percent of patients with newly diagnosed low back pain are prescribed opioids and lower back pain currently contributes 52 percent to the overall opioid prescription rate.\n\nIn addition to boosting opioids distribution, alternative, invasive lower back pain treatments can significantly impact healthcare spending.\n\nIt is not new information that physical therapy and chiropractic care are effective, lower cost alternatives to spinal imaging or surgery. However, payers are still in the process of adopting the method.\n\nTo counteract the high-cost, high-risk potential of using opioids to treat back pain, UHC created a benefit that does not rely on medication or technology but rather on physical therapy and chiropractic care.\n\nThe benefit allows eligible employers to offer physical therapist and chiropractor visits with no out-of-pocket costs. Members who already receive physical therapist and chiropractic care benefits under UHC?s employer-sponsored health plans and who have maxed out their visits will not receive additional visits under this benefit.\n\nHowever, for those who still have visits to use and who choose physical therapy or chiropractic care over other forms of treatment, the copay or deductible for those visits will be waived and they will receive three visits at no cost.\n\nUHC has high expectations for the fiscal and physical impacts of this benefit.\n\nAccording to UHC?s analysis, the health payer expects that by 2021, opioid use will decrease by 19 percent. Spinal imaging test frequency and spinal surgeries will be reduced by 22 percent and 21 percent, respectively. In addition to these specific goals, UHC hopes to see a decrease in the overall cost of spinal care.\n\nThe same OptumLabs study demonstrated that UHC?s expectations are not without precedent.\n\nThe study looked at the correlation between out-of-pocket costs and patient utilization of noninvasive treatments. Researchers discovered that members whose copay was over $30 were a little under 30 percent less likely to choose physical therapy as opposed to more invasive treatments.\n\nAn American Journal of Managed Care study in June 2019 found that patients with high deductibles, typically over $1,000, were less likely to visit physical therapy.\n\nEligible employers may be brand new or renewing their membership. They must be fully insured and over 51 or more employees strong. The benefit is currently available in Connecticut, Florida, Georgia, New York, and North Carolina.\n\nHowever, UHC plans to expand the benefit from 2020 into 2021. By the end of this expansion period, the benefit will also be available to self-funded employers and organizations with an employee population between 2 and 50. The benefit will span ten states, primarily in the southeast.\n\n?This new benefit design may help encourage people with low back pain to get the right care at the right time and in the right setting, helping expand access to evidence-based and more affordable treatments,? said Docimo.
## 3 The Safety of Chiropractic Adjustments\nBy Lana Barhum\nMedically reviewed by Richard N. Fogoros, MD\nUpdated on January 31, 2020\nOrthopedics\nMore in Orthopedics\n\n Home Office Ergonomics\n Sprains & Strains\n Fractures & Broken Bones\n Physical Therapy\n Orthopedic Surgery\n Osteoporosis\n Pediatric Orthopedics\n Sports Injuries\n\nView All\nIn This Article\n\n Chiropractic Adjustment\n What Research Shows\n Safety\n\nChiropractic adjustment, also called spinal manipulation, is a procedure done by a chiropractor using the hands or small instruments to apply controlled force to a spinal joint. The goal is to improve spinal motion and physical function of the entire body. Chiropractic adjustment is safe when performed by someone who is properly trained and licensed to practice chiropractic care. Complications are rare, but they are possible. Learn more about both the benefits and risks.\nChiropractic adjustment\nVerywell / Brianna Gilmartin \nChiropractic Adjustment\n\nOne of the most important reasons people seek chiropractic care is because it is a completely drug-free therapy. Someone dealing with joint pain, back pain, or headaches might consider visiting a chiropractor.\n\nThe goal of chiropractic adjustment is to place the body into a proper position so the body can heal itself. Treatments are believed to reduce stress on the immune system, reducing the potential for disease. Chiropractic care aims to address the entire body, including a person?s ability to move, perform, and even think.\nWhat Research Shows\n\nMany people wonder how helpful chiropractic care is in treating years of trauma and poor posture. There have been numerous studies showing the therapeutic benefits of chiropractic care.\nSciatica\n\nSciatica is a type of pain affecting the sciatic nerve, the large nerve extending from the low back down the back of the legs. Other natural therapies don?t always offer relief and most people want to avoid steroid injections and surgery, so they turn to chiropractic care.\n\nA double-blind trial reported in the Spine Journal compared active and simulated chiropractic manipulations in people with sciatic nerve pain. Active manipulations involved the patient laying down and receiving treatment from a chiropractor. Stimulated manipulations involved electrical muscle stimulation with electrodes placed on the skin to send electrical pulses to different parts of the body.\n\nThe researchers determined active manipulation offered more benefits than stimulated. The people who received active manipulations experienced fewer days of moderate or severe pain and other sciatica symptoms. They also reported no adverse effects.\nNeck Pain\n\nOne study reported in the Annals of Internal Medicine looked at different therapies for treating neck pain. They divided 272 study participants into three groups: one that received spinal manipulation from a chiropractic doctor, a second group given over-the-counter (OTC) pain relievers, narcotics, and muscle relaxers, and a third group who did at-home exercises. \n\nAfter 12 weeks, patients reported a 75% pain reduction, with the chiropractic treatment group achieving the most improvement. About 57% of the chiropractic group achieved pain reduction, while 48% received pain reduction from exercising, and 33% from medication.\n\nAfter one year, 53% of the drug-free groups continued to report pain relief compared to only 38% of those taking pain medications. \nHeadaches\n\nCervicogenic headaches and migraines are commonly treated by chiropractors. Cervicogenic headaches are often called secondary headaches because pain is usually referred from another source, usually the neck. Migraine headaches cause severe, throbbing pain and are generally experienced on one side of the head. There are few non-medicinal options for managing both types of chronic headaches.\n\nResearch reported in the Journal of Manipulative and Physiological Therapeutics suggests chiropractic care, specifically spinal manipulation, can improve migraines and cervicogenic headaches. \nFrozen Shoulder\n\nFrozen shoulder affects the shoulder joint and involves pain and stiffness that develops gradually and gets worse. Frozen shoulder can be quite painful, and treatment involves preserving as much range of motion in the shoulder as possible and managing pain.\n\nA clinical trial reported in the Journal of Chiropractic Medicine described how patients suffering from frozen shoulder responded to chiropractic treatment. Of the 50 patients, 16 completely recovered, 25 showed a 75 to 90% improvement, and eight showed a 50 to 75% improvement. Only one person showed zero to 50% improvement. The researchers concluded most people can get improvement by treating frozen shoulder with chiropractic treatment.\nPreventing Need for Surgery\n\nChiropractic care may reduce the need for back surgery. Guidelines reported in the Journal of the American Medical Association suggest that it's reasonable for people suffering from back pain to try spinal manipulation before deciding on surgical intervention.\nLow Back Pain\n\nStudies have shown chiropractic care, including spinal manipulation, can provide relief from mild to moderate low back pain. In fact, spinal manipulation may work as well as other standard treatments, including pain-relief medications.\n\nA 2011 review of 26 clinical trials looked at the effectiveness of different treatments for chronic low back pain. What they found was that spinal manipulation is just as effective as other treatments for reducing back pain and improving function.\nSafety\n\n\nWhen chiropractors are correctly trained and licensed, chiropractic care is safe. Mild side effects are to be expected and include temporary soreness, stiffness, and tenderness in the treated area. However, you still want to do your research. Ask for a referral from your doctor. Look at the chiropractor?s website, including patient reviews. Meet with the chiropractor to discuss his or her treatment practices and ask about possible adverse effects related to treatment.\n\nIf you decide a chiropractor isn?t for you, consider seeing an osteopathic doctor. Osteopaths are fully licensed doctors who can practice all areas of medicine. They have received special training on the musculoskeletal system, which includes manual readjustments, myofascial release and other physical manipulation of bones and muscle tissues.
## 4 Advanced Chiropractic Relief: 8 Key Benefits of Chiropractor Care\n\nAre you one of the 50 million Americans who suffer from chronic pain? If so you?re probably intimately familiar with the feeling of pure desperation that can arise from an inability to find relief.\n\nIn addition to physical issues, chronic pain can cause anxiety, depression, and more. However, there could be a light at the end of the tunnel. Many people are finding advanced chiropractic relief that is completely changing their lives.\n\nYour body is a world in itself. At this very moment, more than a million chemical reactions are taking place in your body. It manufactures energy, it regulates your heartbeat, your breathing and it regenerates and heals itself. Everything takes place without your conscious knowledge, without you controlling it voluntarily. The master system that controls it all is your nervous system.\n\nThe nervous system is made out of your brain, spinal cord and all your nerves.\n\nThe energy that flows through your nervous system in your body is like electricity. In order to have that electric flow normally and freely, we need to have a well functioning spine. Whenever you have disruption of that flow, disease happens. That would be the case when your spine is misaligned or is not moving properly.\n\nDid you know that 90% of stimulation and nutrition to the brain is generated by the movement of the spine?\n\nThe more mechanically distorted a person is, the less energy is available for thinking, metabolism and healing.\n\nThis is why it is so important to have a healthy spine, a proper posture, to exercise, to eat properly ? all of it truly matters for your quality of life.\nChiropractors localize the areas of your spine that do not move properly ? referred to as vertebral subluxations ? and adjust them with a specific high speed, but yet gentle, thrust to improve spinal motion.\n\nWant to learn about some of the ways chiropractic care can help you? Keep reading for insight into some of the key benefits of seeing a chiropractor.\n\nThe benefits of chiropractic care are numerous:\n\n1. Lower Blood Pressure\n\nStudies show that chiropractic treatment can lower your blood pressure. Sometimes, this works just as well as a prescription blood pressure medication! This benefit can also last for as long as six months after treatment.\n\nHigh blood pressure can cause an array of serious side effects like nausea, fatigue, dizziness, and anxiety. Sufferers who haven?t found relief should consider consulting with a chiropractor. A chiropractic adjustment may be the solution.\n\nSome studies have shown that chiropractic adjustments can also help patients who are suffering from low blood pressure.\n\n2. Reduced Inflammation\n\nIn many cases, joint issues, pain, and tension are caused by inflammation in the body. Chiropractic adjustments can reduce inflammation.\n\nThis leads to relief of muscle tension, chronic back pain, and joint pain. These adjustments can sometimes also slow the progression of inflammation-related diseases, like arthritis.\n\n3. Better Sleep\n\nPatients who receive chiropractic adjustments report a significant improvement in their sleep patterns. If you regularly suffer from insomnia, visiting a chiropractor regularly may help. Also, when you experience pain relief, this will help you get a restful night?s sleep.\n\n4. Digestive Relief\n\nChiropractors often give nutritional advice as part of their services. However, this isn?t the only way that they provide patients with digestive relief.\n\nAdjusting the thoraco-lumbar spine restores the neurological function of your digestive system. Regular adjustments can help with chronic digestive issues.\n\n5. Stress Release\n\nEveryday life can cause muscle cramping, inflammation, and more. When you?re sore from working at a computer, heavy lifting, or just dealing with emotional stress, a chiropractic adjustment can help. This leads to greater comfort and advanced pain relief.\n\n6. Improvement of Neurological Conditions\n\nA chiropractic adjustment can also increase blood flow to the brain and increase the flow of cerebral spinal fluid. This means that patients suffering from neurological conditions like epilepsy and multiple sclerosis can significantly benefit from regular adjustments.\n\nThis is a relatively new area of study, but the potential is huge. Those suffering from these conditions will want to do some research. It?s important to find the best chiropractor in their area with experience dealing with these specific types of cases.\n\n7. Chiropractic care can improve communication from your brain to your muscles\n\nResearch seems to show that chiropractic care can improve your brain-body communication, helping your brain to be more aware of what is going on in the body so it can control your body better.\n\nBetter health, more energy and vitality are some of the positive effects of getting your spine adjusted. It sets your vertebrae back into motion freeing up the energy that travels through your nerves.\n\nChiropractic care is a partnership. The results patients want is a combination of what the chiropractor does and what the patient does.\n\nThere are many good things that can be changed and improved for a better lifestyle: exercise, good nutrition, good mental attitude and spinal adjustments.\n\nYour whole body will work better by having your nervous system free of interference. That is the essence of chiropractic care and is designed for you and your family.\n\n8. Pain Relief\n\nPerhaps the most well-known benefit of going to a chiropractor is pain relief. Adjustments can help with a huge array of painful conditions including the following.\n\nNeck and Lower Back Pain\n\nAdjustments are the most effective non-invasive pain relief method for this type of pain. They may help patients avoid having to take prescription pain management drugs.\n\nSciatica\n\nTreatments help relieve pressure on the nerve. This results in less severe pain that lasts for a fewer number of days.\n\nHeadaches\n\nChiropractic adjustments help headaches and migraines. They do this by treating back misalignment, muscle tension, and stress. Cervical spine manipulation was associated with significant improvement in headache outcomes in trials involving patients with neck pain and/or neck dysfunction and headache.\n\nChronic headaches can result from the abnormal positioning of the head and can be worsened from neck pressure and movement. Chiropractic removes the interference whether it may be from the distant muscle tightness in the back causing strain on your spine or an abnormal lordotic cervical curve and moving vertebrae.\nChiropractic care can reduce the duration of headaches, lower their intensity when they do occur and limit the frequency of their occurrence all together.\n\nMenstrual cramps\n\nChiropractic treatment removes tension from the pelvis and sacrum. It also regulates the neurological function communicating with the reproductive organs. Adjustments can also relieve the bloating, cramping, and pain associated with menstrual cramps\n\nAnyone who has tried traditional medical treatments and has been unable to find pain relief should experiment with chiropractic care. More often than not, you?ll be pleasantly surprised!\n\nBonus: Advanced Chiropractic Relief\n\nIn addition to the benefits listed above, adjustments can bring advanced chiropractic relief for a wide variety of other conditions as well as overall life improvement. A few examples include:\n\nScoliosis ? adjustments have shown to help with the pain, reduced range of motion, abnormal posture, and even difficulty breathing caused by this abnormal curvature of the spine\n\nVertigo ? an adjustment can help realign and balance the spine, thereby reducing the dizziness, nausea, and disorientation caused by vertigo\n\nSinus and allergy relief ? adjusting the upper cervical spine can help drain the sinuses and provide immediate and lasting relief from both long-term and seasonal allergies\n\nExpectant mothers ? women can experience relief from pain and morning sickness and are better able to maintain proper posture during and after pregnancy\n\nChildren?s issues ? treatments have been shown to help children with acid reflux, cholic, and ear infections\nAthletic performance ? the reduction in pain and inflammation is particularly beneficial for professional and amateur athletes\n\nStimulates the immune system ? chiropractic care helps to boost the immune system, speeding up the healing process following illnesses or injuries. One of the most important studies showing the positive effect chiropractic care can have on the immune system and general health was performed by Ronald Pero, Ph.D., chief of cancer prevention research at New York?s Preventive Medicine Institute and professor of medicine at New York University. Dr. Pero measured the immune systems of people under chiropractic care as compared to those in the general population and those with cancer and other serious diseases.\n\nIn his initial three-year study of 107 individuals who had been under chiropractic care for five years or more, the chiropractic patients were found to have a 200% greater immune competence than people who had not received chiropractic care, and 400% greater immune competence than people with cancer and other serious diseases. The immune system superiority of those under chiropractic care did not diminish with age.\n\nDr. Pero stated: ?When applied in a clinical framework, I have never seen a group other than this chiropractic group to experience a 200% increase over the normal patients. This is why it is so dramatically important. We have never seen such a positive improvement in a group.?\n\nAs you can see, there are almost limitless benefits to seeking chiropractic treatment. If you haven?t tried it yet, what are you waiting for?\n\nThere?s no need to accept pain and discomfort as a normal part of life. You have nothing to lose and everything to gain, so it only makes sense to find out more about this possibly life-changing approach to improving your health and wellness.
## 5 Heading to the spa can be a pampering treat, but it can also be a huge boost to your health and wellness! Massage therapy can relieve all sorts of ailments ? from physical pain, to stress and anxiety. People who choose to supplement their healthcare regimen with regular massages will not only enjoy a relaxing hour or two at the spa, but they will see the benefits carry through the days and weeks after the appointment!\n\n1\n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\nThese are the 10 most common benefits reported from massage therapy:\n\n1. Reduce Stress\n\nA relaxing day at the spa is a great way to unwind and de-stress. However, clients are sure to notice themselves feeling relaxed and at ease for days and even weeks after their appointments!\n\n \n\n2. Improve Circulation\n\nLoosening muscles and tendons allows increased blood flow throughout the body. Improving your circulation can have a number of positive effects on the rest of your body, including reduced fatigue and pain management!\n\n \n\n3. Reduce Pain\n\nMassage therapy is great for working out problem areas like lower back pain and chronic stiffness. A professional therapist will be able to accurately target the source of your pain and help achieve the perfect massage regimen.\n\n \n\n3\n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n4. Eliminate Toxins\n\nStimulating the soft tissues of your body will help to release toxins through your blood and lymphatic systems.\n\n \n\n5. Improve Flexibility\n\nMassage therapy will loosen and relax your muscles, helping your body to achieve its full range of movement potential.\n\n \n\n6. Improve Sleep\n\nA massage will encourage relaxation and boost your mood. Going to bed with relaxed and loosened muscles promotes more restful sleep, and you?ll feel less tired in the morning!\n\n \n\n7. Enhance Immunity\n\nStimulation of the lymph nodes re-charges the body?s natural defense system.\n\n \n\n8. Reduce Fatigue\n\nMassage therapy is known to boost mood and promote better quality sleep, thus making you feel more rested and less worn-out at the end of the day.\n\n \n\n2\n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n9. Alleviate Depression and Anxiety\n\nMassage therapy can help to release endorphins in your body, helping you to feel happy, energized, and at ease.\n\n \n\n10. Reduce post-surgery and post-injury swelling\n\nA professional massage is a great way to safely deal with a sports injury or post-surgery rehabilitation.\n\nDo you think that massage therapy could help you find relief in any of these areas? What improvements would you like to see in your health? Contact us today with your questions about massage therapy and see how we can help you get on the path to improved health and wellness!
## Source
## 1 https://coremedicalohio.com/benefits-of-long-term-chiropractic-care/?utm_source=ReviveOldPost&utm_medium=social&utm_campaign=ReviveOldPost
## 2 https://healthpayerintelligence.com/news/unitedhealthcare-combats-opioid-crisis-with-non-opioid-benefits
## 3 https://www.verywellhealth.com/is-chiropractic-adjustment-safe-4588279
## 4 https://hafkeychiropractic.com/advanced-chiropractic-relief-8-key-benefits-of-chiropractor-care/
## 5 https://www.urbannirvana.com/10-benefits-massage-therapy/
## Topic InternetSearch Contraindications
## 1 chiropractic benefits <NA> <NA>
## 2 chiropractic benefits <NA> <NA>
## 3 chiropractic benefits <NA> <NA>
## 4 chiropractic benefits <NA> <NA>
## 5 massage benefits google <NA>
## risksAdverseEffects
## 1 <NA>
## 2 <NA>
## 3 Risks and side effects associated with chiropractic adjustments may include:\n\n temporary headaches\n fatigue after treatment\n discomfort in parts of the body that were treated\n\nRare but serious risks associated with chiropractic adjustment include:\n\n stroke\n cauda equina syndrome, a condition involving pinched nerves in the lower part of the spinal canal\n worsening of herniated disks (although research isn't conclusive)\n\nIn addition to effectiveness, research has focused on the safety of chiropractic treatments, mainly spinal manipulation. \n\nOne 2017 review of 250 articles looked at serious adverse events and benign events associated with chiropractic care. Based on the evidence the researchers reviewed, serious adverse events accounted for one out of every two million spinal manipulations to 13 per 10,00 patients. Serious adverse events included spinal or neurological problems and cervical arterial strokes (dissection of any of the arteries in the neck).\n\nBenign events were more common and included more pain and higher levels of neck problems, but most were short-term problems.\n\nThe researchers confirmed serious adverse events were rare and often related to other preexisting conditions, while benign events are more common. However, the reasons for any types of adverse events are unknown.\n\nA second 2017 review looked 118 articles and found frequently described adverse events include stroke, headache and vertebral artery dissection (cervical arterial stroke). Forty-six percent of the reviews determined that spinal manipulation was safe, while 13% expressed concern of harm. The remaining studies were unclear or neutral. While the researchers did not offer an overall conclusion, they determined spinal manipulation can significantly be helpful, and some risk does exist.\nA Word From Verywell
## 4 <NA>
## 5 <NA>def count_punct(text):
count=sum([1 for char in text if char in string.punctuation])
return round(count/(len(text)-text.count(" ")),3)*100data['body_length']=data['Document'].apply(lambda x: len(x)-x.count(" "))
data['punct%']= data['Document'].apply(lambda x: count_punct(x))
def clean_text(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=[ps.stem(word) for word in tokens if word not in stopwords]#these are lists for the count vectorizer
return text
def lemmatize(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=[wn.lemmatize(word) for word in tokens if word not in stopwords]#these are lists for the count vectorizer
return text
data['Cleaned_text']=data['Document'].apply(lambda x: clean_text(x))
data['Lemmatized']=data['Document'].apply(lambda x: lemmatize(x))
data.head()## Document ... Lemmatized
## 0 Chiropractic adjustments and treatments serve ... ... [chiropractic, adjustment, treatment, serve, n...
## 1 \r\nUnitedHealthcare Combats Opioid Crisis wit... ... [, unitedhealthcare, combat, opioid, crisis, n...
## 2 The Safety of Chiropractic Adjustments\r\nBy ... ... [, safety, chiropractic, adjustment, lana, bar...
## 3 Advanced Chiropractic Relief: 8 Key Benefits o... ... [advanced, chiropractic, relief, 8, key, benef...
## 4 Heading to the spa can be a pampering treat, b... ... [heading, spa, pampering, treat, also, huge, b...
##
## [5 rows x 10 columns]data.to_csv('dataCleanLemm.csv')
#DATA = pd.read_csv('dataCleanLemm.csv', encoding='unicode_escape')DATA <- read.csv('dataCleanLemm.csv', sep=',', header=TRUE, na.strings=c('',' ','NA'), row.names=1)
colnames(DATA)## [1] "Document" "Source" "Topic"
## [4] "InternetSearch" "Contraindications" "RisksSideEffects"
## [7] "body_length" "punct." "Cleaned_text"
## [10] "Lemmatized"head(DATA,2)## Document
## 0 Chiropractic adjustments and treatments serve the needs of millions of people around the world.\n\n \n\nAdjustments offer effective, non-invasive and cost-effective solutions to neck and back pain, as well as a myriad of other medical issues.\n\nHave you ever stopped to wonder how many of us suffer from neck and back stiffness or pain?\n\nApart from the obvious discomfort, simple daily tasks such as driving a car, crossing a busy street and picking things up from the floor can become all too challenging for individuals experiencing such pain.\n\nAs anyone who has experienced pain would know, having restricted movement can be debilitating and unfortunately, our busy world doesn?t allow for us to stop.\n\n \nSome of the benefits of long-term chiropractic care include:\n\n Chiropractors can identify mechanical issues that cause spine-related pain and offer a series of adjustments that provide near immediate relief. Following appointments, patients often report feeling their symptoms noticeably better.\n When a chiropractor performs an adjustment, they can help restore movement in joints that have ?locked up?. This becomes possible as treatment allows muscles surrounding joints to relax, thereby reducing joint stiffness.\n Many factors affect health, including exercise patterns, nutrition, sleep, heredity and the environment in which we live. Rather than just treat symptoms of the disease, chiropractic care focuses on a holistic approach to naturally maintain health and resist disease.\n Chiropractic adjustments help restore normal function and movement to the entire body. Many patients report an improvement in their ability to move with efficiency and strength.\n Many patients find delight in the results chiropractic adjustments have on old and chronic injuries. Whether an injury is, in fact, new or old, chiropractic care can help reduce pain, restore mobility and provide quick pain relief to all joints in the body. Such care can help maintain better overall health and thus faster recovery time.\n\nHave you ever noticed that when you are in pain and unable to perform regular or favorite activities, it can put a strain on emotional and mental well-being?\n\nFor example, the increased stress from not being able to properly perform a paid job. This, in turn, can have a negative impact on physical health with increases in heart rate and blood pressure. The domino effect often continues with sleep becoming disturbed, with resulting lethargy and tiredness during the day. Does anyone really feel up to exercising in this state?\n\nChiropractic care is a natural method of healing the body?s communication system and never relies on the use of pharmaceutical drugs or invasive surgery.\n\nDoctors of Chiropractic work collaboratively with other healthcare professionals. Should your condition require the attention of another healthcare profession, that recommendation or referral will be made.\n\n
## 1 \nUnitedHealthcare Combats Opioid Crisis with Non-Opioid Benefits\nPhysical therapy and chiropractic care can prevent or reduce expensive, invasive spinal procedures, such as imaging or surgery, to reduce opioid use and cut costs.\nUnitedHealthcare, opioid, physical therapy, healthcare spending\n\nSource: Thinkstock\nShare on Twitter\n\nBy Kelsey Waddill\n\nOctober 29, 2019 - UnitedHealthcare (UHC) is combatting the opioid epidemic and high healthcare costs with new physical therapy and chiropractic care benefits to prevent, delay, or in some cases substitute for invasive spinal procedures.\n\n?With millions of Americans experiencing low back pain currently or at some point during their lifetimes, we believe this benefit design will help make a meaningful difference by improving health outcomes while reducing costs,? said Anne Docimo, MD, UnitedHealthcare chief medical officer.\n\nLower back pain is in part responsible for sustaining the opioid epidemic and also increases healthcare costs.\nDig Deeper\n\n How Major Payers Provide Substance Abuse Care for Opioid Misuse\n Opioid Overdoses Fall by 2% from 2017 to 2018, CDC Reports\n Physical Therapy, Chiropractic Back Care Cut Opioid Use, Costs\n\nAlthough opioid overdoses fell by two percent from 2017 to 2018 and a legal battles aim to hold pharmaceutical companies accountable, there is no end in sight for the opioid epidemic. Industry professionals are still grappling with the balance between cutting opioid prescriptions will working to reduce patient pain.\n\nCommon conditions such as low back pain bolster the epidemic?s presence, with clinicians still prescribing the opioids against best practice recommendations. According to a recent OptumLabs study, 9 percent of patients with newly diagnosed low back pain are prescribed opioids and lower back pain currently contributes 52 percent to the overall opioid prescription rate.\n\nIn addition to boosting opioids distribution, alternative, invasive lower back pain treatments can significantly impact healthcare spending.\n\nIt is not new information that physical therapy and chiropractic care are effective, lower cost alternatives to spinal imaging or surgery. However, payers are still in the process of adopting the method.\n\nTo counteract the high-cost, high-risk potential of using opioids to treat back pain, UHC created a benefit that does not rely on medication or technology but rather on physical therapy and chiropractic care.\n\nThe benefit allows eligible employers to offer physical therapist and chiropractor visits with no out-of-pocket costs. Members who already receive physical therapist and chiropractic care benefits under UHC?s employer-sponsored health plans and who have maxed out their visits will not receive additional visits under this benefit.\n\nHowever, for those who still have visits to use and who choose physical therapy or chiropractic care over other forms of treatment, the copay or deductible for those visits will be waived and they will receive three visits at no cost.\n\nUHC has high expectations for the fiscal and physical impacts of this benefit.\n\nAccording to UHC?s analysis, the health payer expects that by 2021, opioid use will decrease by 19 percent. Spinal imaging test frequency and spinal surgeries will be reduced by 22 percent and 21 percent, respectively. In addition to these specific goals, UHC hopes to see a decrease in the overall cost of spinal care.\n\nThe same OptumLabs study demonstrated that UHC?s expectations are not without precedent.\n\nThe study looked at the correlation between out-of-pocket costs and patient utilization of noninvasive treatments. Researchers discovered that members whose copay was over $30 were a little under 30 percent less likely to choose physical therapy as opposed to more invasive treatments.\n\nAn American Journal of Managed Care study in June 2019 found that patients with high deductibles, typically over $1,000, were less likely to visit physical therapy.\n\nEligible employers may be brand new or renewing their membership. They must be fully insured and over 51 or more employees strong. The benefit is currently available in Connecticut, Florida, Georgia, New York, and North Carolina.\n\nHowever, UHC plans to expand the benefit from 2020 into 2021. By the end of this expansion period, the benefit will also be available to self-funded employers and organizations with an employee population between 2 and 50. The benefit will span ten states, primarily in the southeast.\n\n?This new benefit design may help encourage people with low back pain to get the right care at the right time and in the right setting, helping expand access to evidence-based and more affordable treatments,? said Docimo.
## Source
## 0 https://coremedicalohio.com/benefits-of-long-term-chiropractic-care/?utm_source=ReviveOldPost&utm_medium=social&utm_campaign=ReviveOldPost
## 1 https://healthpayerintelligence.com/news/unitedhealthcare-combats-opioid-crisis-with-non-opioid-benefits
## Topic InternetSearch Contraindications RisksSideEffects
## 0 chiropractic benefits <NA> <NA> <NA>
## 1 chiropractic benefits <NA> <NA> <NA>
## body_length punct.
## 0 2497 2.3
## 1 4055 2.4
## Cleaned_text
## 0 ['chiropract', 'adjust', 'treatment', 'serv', 'need', 'million', 'peopl', 'around', 'world', 'adjust', 'offer', 'effect', 'noninvas', 'costeffect', 'solut', 'neck', 'back', 'pain', 'well', 'myriad', 'medic', 'issu', 'ever', 'stop', 'wonder', 'mani', 'us', 'suffer', 'neck', 'back', 'stiff', 'pain', 'apart', 'obviou', 'discomfort', 'simpl', 'daili', 'task', 'drive', 'car', 'cross', 'busi', 'street', 'pick', 'thing', 'floor', 'becom', 'challeng', 'individu', 'experienc', 'pain', 'anyon', 'experienc', 'pain', 'would', 'know', 'restrict', 'movement', 'debilit', 'unfortun', 'busi', 'world', 'doesnt', 'allow', 'us', 'stop', 'benefit', 'longterm', 'chiropract', 'care', 'includ', 'chiropractor', 'identifi', 'mechan', 'issu', 'caus', 'spinerel', 'pain', 'offer', 'seri', 'adjust', 'provid', 'near', 'immedi', 'relief', 'follow', 'appoint', 'patient', 'often', 'report', 'feel', 'symptom', 'notic', 'better', 'chiropractor', 'perform', 'adjust', 'help', 'restor', 'movement', 'joint', 'lock', 'becom', 'possibl', 'treatment', 'allow', 'muscl', 'surround', 'joint', 'relax', 'therebi', 'reduc', 'joint', 'stiff', 'mani', 'factor', 'affect', 'health', 'includ', 'exercis', 'pattern', 'nutrit', 'sleep', 'hered', 'environ', 'live', 'rather', 'treat', 'symptom', 'diseas', 'chiropract', 'care', 'focus', 'holist', 'approach', 'natur', 'maintain', 'health', 'resist', 'diseas', 'chiropract', 'adjust', 'help', 'restor', 'normal', 'function', 'movement', 'entir', 'bodi', 'mani', 'patient', 'report', 'improv', 'abil', 'move', 'effici', 'strength', 'mani', 'patient', 'find', 'delight', 'result', 'chiropract', 'adjust', 'old', 'chronic', 'injuri', 'whether', 'injuri', 'fact', 'new', 'old', 'chiropract', 'care', 'help', 'reduc', 'pain', 'restor', 'mobil', 'provid', 'quick', 'pain', 'relief', 'joint', 'bodi', 'care', 'help', 'maintain', 'better', 'overal', 'health', 'thu', 'faster', 'recoveri', 'time', 'ever', 'notic', 'pain', 'unabl', 'perform', 'regular', 'favorit', 'activ', 'put', 'strain', 'emot', 'mental', 'wellb', 'exampl', 'increas', 'stress', 'abl', 'properli', 'perform', 'paid', 'job', 'turn', 'neg', 'impact', 'physic', 'health', 'increas', 'heart', 'rate', 'blood', 'pressur', 'domino', 'effect', 'often', 'continu', 'sleep', 'becom', 'disturb', 'result', 'lethargi', 'tired', 'day', 'anyon', 'realli', 'feel', 'exercis', 'state', 'chiropract', 'care', 'natur', 'method', 'heal', 'bodi', 'commun', 'system', 'never', 'reli', 'use', 'pharmaceut', 'drug', 'invas', 'surgeri', 'doctor', 'chiropract', 'work', 'collabor', 'healthcar', 'profession', 'condit', 'requir', 'attent', 'anoth', 'healthcar', 'profess', 'recommend', 'referr', 'made', '']
## 1 ['', 'unitedhealthcar', 'combat', 'opioid', 'crisi', 'nonopioid', 'benefit', 'physic', 'therapi', 'chiropract', 'care', 'prevent', 'reduc', 'expens', 'invas', 'spinal', 'procedur', 'imag', 'surgeri', 'reduc', 'opioid', 'use', 'cut', 'cost', 'unitedhealthcar', 'opioid', 'physic', 'therapi', 'healthcar', 'spend', 'sourc', 'thinkstock', 'share', 'twitter', 'kelsey', 'waddil', 'octob', '29', '2019', 'unitedhealthcar', 'uhc', 'combat', 'opioid', 'epidem', 'high', 'healthcar', 'cost', 'new', 'physic', 'therapi', 'chiropract', 'care', 'benefit', 'prevent', 'delay', 'case', 'substitut', 'invas', 'spinal', 'procedur', 'million', 'american', 'experienc', 'low', 'back', 'pain', 'current', 'point', 'lifetim', 'believ', 'benefit', 'design', 'help', 'make', 'meaning', 'differ', 'improv', 'health', 'outcom', 'reduc', 'cost', 'said', 'ann', 'docimo', 'md', 'unitedhealthcar', 'chief', 'medic', 'offic', 'lower', 'back', 'pain', 'part', 'respons', 'sustain', 'opioid', 'epidem', 'also', 'increas', 'healthcar', 'cost', 'dig', 'deeper', 'major', 'payer', 'provid', 'substanc', 'abus', 'care', 'opioid', 'misus', 'opioid', 'overdos', 'fall', '2', '2017', '2018', 'cdc', 'report', 'physic', 'therapi', 'chiropract', 'back', 'care', 'cut', 'opioid', 'use', 'cost', 'although', 'opioid', 'overdos', 'fell', 'two', 'percent', '2017', '2018', 'legal', 'battl', 'aim', 'hold', 'pharmaceut', 'compani', 'account', 'end', 'sight', 'opioid', 'epidem', 'industri', 'profession', 'still', 'grappl', 'balanc', 'cut', 'opioid', 'prescript', 'work', 'reduc', 'patient', 'pain', 'common', 'condit', 'low', 'back', 'pain', 'bolster', 'epidem', 'presenc', 'clinician', 'still', 'prescrib', 'opioid', 'best', 'practic', 'recommend', 'accord', 'recent', 'optumlab', 'studi', '9', 'percent', 'patient', 'newli', 'diagnos', 'low', 'back', 'pain', 'prescrib', 'opioid', 'lower', 'back', 'pain', 'current', 'contribut', '52', 'percent', 'overal', 'opioid', 'prescript', 'rate', 'addit', 'boost', 'opioid', 'distribut', 'altern', 'invas', 'lower', 'back', 'pain', 'treatment', 'significantli', 'impact', 'healthcar', 'spend', 'new', 'inform', 'physic', 'therapi', 'chiropract', 'care', 'effect', 'lower', 'cost', 'altern', 'spinal', 'imag', 'surgeri', 'howev', 'payer', 'still', 'process', 'adopt', 'method', 'counteract', 'highcost', 'highrisk', 'potenti', 'use', 'opioid', 'treat', 'back', 'pain', 'uhc', 'creat', 'benefit', 'reli', 'medic', 'technolog', 'rather', 'physic', 'therapi', 'chiropract', 'care', 'benefit', 'allow', 'elig', 'employ', 'offer', 'physic', 'therapist', 'chiropractor', 'visit', 'outofpocket', 'cost', 'member', 'alreadi', 'receiv', 'physic', 'therapist', 'chiropract', 'care', 'benefit', 'uhc', 'employersponsor', 'health', 'plan', 'max', 'visit', 'receiv', 'addit', 'visit', 'benefit', 'howev', 'still', 'visit', 'use', 'choos', 'physic', 'therapi', 'chiropract', 'care', 'form', 'treatment', 'copay', 'deduct', 'visit', 'waiv', 'receiv', 'three', 'visit', 'cost', 'uhc', 'high', 'expect', 'fiscal', 'physic', 'impact', 'benefit', 'accord', 'uhc', 'analysi', 'health', 'payer', 'expect', '2021', 'opioid', 'use', 'decreas', '19', 'percent', 'spinal', 'imag', 'test', 'frequenc', 'spinal', 'surgeri', 'reduc', '22', 'percent', '21', 'percent', 'respect', 'addit', 'specif', 'goal', 'uhc', 'hope', 'see', 'decreas', 'overal', 'cost', 'spinal', 'care', 'optumlab', 'studi', 'demonstr', 'uhc', 'expect', 'without', 'preced', 'studi', 'look', 'correl', 'outofpocket', 'cost', 'patient', 'util', 'noninvas', 'treatment', 'research', 'discov', 'member', 'whose', 'copay', '30', 'littl', '30', 'percent', 'less', 'like', 'choos', 'physic', 'therapi', 'oppos', 'invas', 'treatment', 'american', 'journal', 'manag', 'care', 'studi', 'june', '2019', 'found', 'patient', 'high', 'deduct', 'typic', '1000', 'less', 'like', 'visit', 'physic', 'therapi', 'elig', 'employ', 'may', 'brand', 'new', 'renew', 'membership', 'must', 'fulli', 'insur', '51', 'employe', 'strong', 'benefit', 'current', 'avail', 'connecticut', 'florida', 'georgia', 'new', 'york', 'north', 'carolina', 'howev', 'uhc', 'plan', 'expand', 'benefit', '2020', '2021', 'end', 'expans', 'period', 'benefit', 'also', 'avail', 'selffund', 'employ', 'organ', 'employe', 'popul', '2', '50', 'benefit', 'span', 'ten', 'state', 'primarili', 'southeast', 'new', 'benefit', 'design', 'may', 'help', 'encourag', 'peopl', 'low', 'back', 'pain', 'get', 'right', 'care', 'right', 'time', 'right', 'set', 'help', 'expand', 'access', 'evidencebas', 'afford', 'treatment', 'said', 'docimo']
## Lemmatized
## 0 ['chiropractic', 'adjustment', 'treatment', 'serve', 'need', 'million', 'people', 'around', 'world', 'adjustment', 'offer', 'effective', 'noninvasive', 'costeffective', 'solution', 'neck', 'back', 'pain', 'well', 'myriad', 'medical', 'issue', 'ever', 'stopped', 'wonder', 'many', 'u', 'suffer', 'neck', 'back', 'stiffness', 'pain', 'apart', 'obvious', 'discomfort', 'simple', 'daily', 'task', 'driving', 'car', 'crossing', 'busy', 'street', 'picking', 'thing', 'floor', 'become', 'challenging', 'individual', 'experiencing', 'pain', 'anyone', 'experienced', 'pain', 'would', 'know', 'restricted', 'movement', 'debilitating', 'unfortunately', 'busy', 'world', 'doesnt', 'allow', 'u', 'stop', 'benefit', 'longterm', 'chiropractic', 'care', 'include', 'chiropractor', 'identify', 'mechanical', 'issue', 'cause', 'spinerelated', 'pain', 'offer', 'series', 'adjustment', 'provide', 'near', 'immediate', 'relief', 'following', 'appointment', 'patient', 'often', 'report', 'feeling', 'symptom', 'noticeably', 'better', 'chiropractor', 'performs', 'adjustment', 'help', 'restore', 'movement', 'joint', 'locked', 'becomes', 'possible', 'treatment', 'allows', 'muscle', 'surrounding', 'joint', 'relax', 'thereby', 'reducing', 'joint', 'stiffness', 'many', 'factor', 'affect', 'health', 'including', 'exercise', 'pattern', 'nutrition', 'sleep', 'heredity', 'environment', 'live', 'rather', 'treat', 'symptom', 'disease', 'chiropractic', 'care', 'focus', 'holistic', 'approach', 'naturally', 'maintain', 'health', 'resist', 'disease', 'chiropractic', 'adjustment', 'help', 'restore', 'normal', 'function', 'movement', 'entire', 'body', 'many', 'patient', 'report', 'improvement', 'ability', 'move', 'efficiency', 'strength', 'many', 'patient', 'find', 'delight', 'result', 'chiropractic', 'adjustment', 'old', 'chronic', 'injury', 'whether', 'injury', 'fact', 'new', 'old', 'chiropractic', 'care', 'help', 'reduce', 'pain', 'restore', 'mobility', 'provide', 'quick', 'pain', 'relief', 'joint', 'body', 'care', 'help', 'maintain', 'better', 'overall', 'health', 'thus', 'faster', 'recovery', 'time', 'ever', 'noticed', 'pain', 'unable', 'perform', 'regular', 'favorite', 'activity', 'put', 'strain', 'emotional', 'mental', 'wellbeing', 'example', 'increased', 'stress', 'able', 'properly', 'perform', 'paid', 'job', 'turn', 'negative', 'impact', 'physical', 'health', 'increase', 'heart', 'rate', 'blood', 'pressure', 'domino', 'effect', 'often', 'continues', 'sleep', 'becoming', 'disturbed', 'resulting', 'lethargy', 'tiredness', 'day', 'anyone', 'really', 'feel', 'exercising', 'state', 'chiropractic', 'care', 'natural', 'method', 'healing', 'body', 'communication', 'system', 'never', 'relies', 'use', 'pharmaceutical', 'drug', 'invasive', 'surgery', 'doctor', 'chiropractic', 'work', 'collaboratively', 'healthcare', 'professional', 'condition', 'require', 'attention', 'another', 'healthcare', 'profession', 'recommendation', 'referral', 'made', '']
## 1 ['', 'unitedhealthcare', 'combat', 'opioid', 'crisis', 'nonopioid', 'benefit', 'physical', 'therapy', 'chiropractic', 'care', 'prevent', 'reduce', 'expensive', 'invasive', 'spinal', 'procedure', 'imaging', 'surgery', 'reduce', 'opioid', 'use', 'cut', 'cost', 'unitedhealthcare', 'opioid', 'physical', 'therapy', 'healthcare', 'spending', 'source', 'thinkstock', 'share', 'twitter', 'kelsey', 'waddill', 'october', '29', '2019', 'unitedhealthcare', 'uhc', 'combatting', 'opioid', 'epidemic', 'high', 'healthcare', 'cost', 'new', 'physical', 'therapy', 'chiropractic', 'care', 'benefit', 'prevent', 'delay', 'case', 'substitute', 'invasive', 'spinal', 'procedure', 'million', 'american', 'experiencing', 'low', 'back', 'pain', 'currently', 'point', 'lifetime', 'believe', 'benefit', 'design', 'help', 'make', 'meaningful', 'difference', 'improving', 'health', 'outcome', 'reducing', 'cost', 'said', 'anne', 'docimo', 'md', 'unitedhealthcare', 'chief', 'medical', 'officer', 'lower', 'back', 'pain', 'part', 'responsible', 'sustaining', 'opioid', 'epidemic', 'also', 'increase', 'healthcare', 'cost', 'dig', 'deeper', 'major', 'payer', 'provide', 'substance', 'abuse', 'care', 'opioid', 'misuse', 'opioid', 'overdoses', 'fall', '2', '2017', '2018', 'cdc', 'report', 'physical', 'therapy', 'chiropractic', 'back', 'care', 'cut', 'opioid', 'use', 'cost', 'although', 'opioid', 'overdoses', 'fell', 'two', 'percent', '2017', '2018', 'legal', 'battle', 'aim', 'hold', 'pharmaceutical', 'company', 'accountable', 'end', 'sight', 'opioid', 'epidemic', 'industry', 'professional', 'still', 'grappling', 'balance', 'cutting', 'opioid', 'prescription', 'working', 'reduce', 'patient', 'pain', 'common', 'condition', 'low', 'back', 'pain', 'bolster', 'epidemic', 'presence', 'clinician', 'still', 'prescribing', 'opioids', 'best', 'practice', 'recommendation', 'according', 'recent', 'optumlabs', 'study', '9', 'percent', 'patient', 'newly', 'diagnosed', 'low', 'back', 'pain', 'prescribed', 'opioids', 'lower', 'back', 'pain', 'currently', 'contributes', '52', 'percent', 'overall', 'opioid', 'prescription', 'rate', 'addition', 'boosting', 'opioids', 'distribution', 'alternative', 'invasive', 'lower', 'back', 'pain', 'treatment', 'significantly', 'impact', 'healthcare', 'spending', 'new', 'information', 'physical', 'therapy', 'chiropractic', 'care', 'effective', 'lower', 'cost', 'alternative', 'spinal', 'imaging', 'surgery', 'however', 'payer', 'still', 'process', 'adopting', 'method', 'counteract', 'highcost', 'highrisk', 'potential', 'using', 'opioids', 'treat', 'back', 'pain', 'uhc', 'created', 'benefit', 'rely', 'medication', 'technology', 'rather', 'physical', 'therapy', 'chiropractic', 'care', 'benefit', 'allows', 'eligible', 'employer', 'offer', 'physical', 'therapist', 'chiropractor', 'visit', 'outofpocket', 'cost', 'member', 'already', 'receive', 'physical', 'therapist', 'chiropractic', 'care', 'benefit', 'uhcs', 'employersponsored', 'health', 'plan', 'maxed', 'visit', 'receive', 'additional', 'visit', 'benefit', 'however', 'still', 'visit', 'use', 'choose', 'physical', 'therapy', 'chiropractic', 'care', 'form', 'treatment', 'copay', 'deductible', 'visit', 'waived', 'receive', 'three', 'visit', 'cost', 'uhc', 'high', 'expectation', 'fiscal', 'physical', 'impact', 'benefit', 'according', 'uhcs', 'analysis', 'health', 'payer', 'expects', '2021', 'opioid', 'use', 'decrease', '19', 'percent', 'spinal', 'imaging', 'test', 'frequency', 'spinal', 'surgery', 'reduced', '22', 'percent', '21', 'percent', 'respectively', 'addition', 'specific', 'goal', 'uhc', 'hope', 'see', 'decrease', 'overall', 'cost', 'spinal', 'care', 'optumlabs', 'study', 'demonstrated', 'uhcs', 'expectation', 'without', 'precedent', 'study', 'looked', 'correlation', 'outofpocket', 'cost', 'patient', 'utilization', 'noninvasive', 'treatment', 'researcher', 'discovered', 'member', 'whose', 'copay', '30', 'little', '30', 'percent', 'le', 'likely', 'choose', 'physical', 'therapy', 'opposed', 'invasive', 'treatment', 'american', 'journal', 'managed', 'care', 'study', 'june', '2019', 'found', 'patient', 'high', 'deductible', 'typically', '1000', 'le', 'likely', 'visit', 'physical', 'therapy', 'eligible', 'employer', 'may', 'brand', 'new', 'renewing', 'membership', 'must', 'fully', 'insured', '51', 'employee', 'strong', 'benefit', 'currently', 'available', 'connecticut', 'florida', 'georgia', 'new', 'york', 'north', 'carolina', 'however', 'uhc', 'plan', 'expand', 'benefit', '2020', '2021', 'end', 'expansion', 'period', 'benefit', 'also', 'available', 'selffunded', 'employer', 'organization', 'employee', 'population', '2', '50', 'benefit', 'span', 'ten', 'state', 'primarily', 'southeast', 'new', 'benefit', 'design', 'may', 'help', 'encourage', 'people', 'low', 'back', 'pain', 'get', 'right', 'care', 'right', 'time', 'right', 'setting', 'helping', 'expand', 'access', 'evidencebased', 'affordable', 'treatment', 'said', 'docimo']from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(data[['Document','body_length','punct%','Cleaned_text','Lemmatized']],data['Topic'],test_size=0.20)from sklearn.feature_extraction.text import CountVectorizercount_vect=CountVectorizer(analyzer=lemmatize)
count_vect_fit=count_vect.fit(X_train['Document'])
count_train=count_vect_fit.transform(X_train['Document'])
count_test=count_vect_fit.transform(X_test['Document'])len(count_vect_fit.get_feature_names())## 4465count_vect_fit.get_feature_names()[200:350]## ['acupressure', 'acupuncture', 'acupuncturist', 'acupuncturists', 'acute', 'ad', 'adapted', 'added', 'addiction', 'adding', 'addition', 'additional', 'additionally', 'address', 'addressed', 'addressing', 'adeline', 'adequate', 'adhesion', 'adjust', 'adjusted', 'adjusting', 'adjustment', 'administered', 'administering', 'administration', 'admission', 'admitted', 'adolescent', 'adopted', 'adopting', 'adult', 'adultery', 'adults1', 'advanced', 'advantage', 'advantageous', 'adverse', 'adversity', 'advertisement', 'advice', 'advise', 'advisory', 'advocate', 'aerobic', 'affair', 'affect', 'affected', 'affecting', 'affiliate', 'affordable', 'afraid', 'afterhours', 'afterward', 'afterwards', 'agakian', 'age', 'agency', 'agephysical', 'agerelated', 'aggression', 'aggressively', 'aging', 'ago', 'agree', 'ahead', 'aid', 'aiding', 'ailment', 'aim', 'air', 'airport', 'ajman', 'aka', 'alchemist', 'alcohol', 'alcoholsoaked', 'alert', 'align', 'aligning', 'alignment', 'alike', 'allergic', 'allergy', 'alleviate', 'alleviates', 'alleviating', 'alleviation', 'allison', 'allow', 'allowing', 'allows', 'almost', 'alone', 'along', 'alongside', 'already', 'also', 'alter', 'altered', 'alternate', 'alternating', 'alternatingly', 'alternative', 'alters', 'although', 'altogether', 'always', 'alzheimers', 'amateur', 'amazing', 'ambulance', 'ameliorating', 'america', 'american', 'among', 'amount', 'amplitude', 'amta', 'amy', 'analgesic', 'analysis', 'analyzed', 'anatomy', 'ancient', 'andor', 'anemia', 'anesthetic', 'angeles', 'anger', 'angle', 'angry', 'animal', 'aniston', 'ankle', 'ann', 'annals', 'anne', 'announced', 'announcing', 'annual', 'annually', 'another', 'answer', 'answered', 'antibiotic', 'antibody', 'anticoagulant', 'antiinflammatory', 'antiviral']count_train_vect=pd.concat([X_train[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(count_train.toarray())],axis=1)
count_test_vect=pd.concat([X_test[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(count_test.toarray())],axis=1)count_train_vect.head()## Document body_length ... 4463 4464
## 0 Hana\r\n@Hana40717041\r\n·\r\n18h\r\nL#Massage... 281 ... 0 0
## 1 enefits of Cold Stone Massage\r\n\r\nOne of th... 2651 ... 0 0
## 2 Available now.. good massage plus service sex.... 68 ... 0 0
## 3 Good morning Ajman.\r\nSpend your day with bea... 157 ... 0 0
## 4 The Benefits of Mental Health According to Sci... 10567 ... 0 1
##
## [5 rows x 4470 columns]from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import precision_recall_fscore_support as score
import timerf=RandomForestClassifier(n_estimators=150, max_depth=None, n_jobs=-1)
start=time.time()
rf_model=rf.fit(count_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=rf_model.predict(count_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 43 cupping benefits cupping benefits
## 53 massage gun benefits massage gun benefits
## 85 Not Professional ER
## 25 massage benefits mental health services benefits
## 55 chiropractic benefits cold stone benefits
## 9 massage benefits massage benefits
## 24 physical therapy benefits physical therapy benefits
## 12 massage benefits massage benefits
## 28 Not Professional mental health services benefits
## 56 cold stone benefits cold stone benefits
## 40 cupping benefits cupping benefits
## 75 Not Professional Not Professional
## 19 Not Professional physical therapy benefits
## 72 Not Professional Not Professional
## 26 massage benefits mental health services benefits
## 61 cold stone benefits cold stone benefits
## 14 massage benefits massage benefits
## 10 massage benefits massage benefitsfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.6666666666666666print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[0 1 0 0 0 0 0 0 0]
## [0 2 0 0 0 0 0 0 0]
## [0 0 0 0 0 0 0 0 0]
## [0 0 1 2 0 0 0 0 0]
## [0 0 0 0 2 0 0 0 0]
## [0 0 0 0 0 4 0 0 0]
## [0 0 0 0 0 0 1 0 0]
## [0 1 0 0 0 2 0 0 0]
## [0 1 0 0 0 0 0 0 1]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## ER 0.00 0.00 0.00 1
## Not Professional 0.40 1.00 0.57 2
## chiropractic benefits 0.00 0.00 0.00 0
## cold stone benefits 1.00 0.67 0.80 3
## cupping benefits 1.00 1.00 1.00 2
## massage benefits 0.67 1.00 0.80 4
## massage gun benefits 1.00 1.00 1.00 1
## mental health services benefits 0.00 0.00 0.00 3
## physical therapy benefits 1.00 0.50 0.67 2
##
## accuracy 0.67 18
## macro avg 0.56 0.57 0.54 18
## weighted avg 0.64 0.67 0.62 18
##
##
## C:\Users\m\Anaconda2\envs\python36\lib\site-packages\sklearn\metrics\classification.py:1437: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.
## 'precision', 'predicted', average, warn_for)
## C:\Users\m\Anaconda2\envs\python36\lib\site-packages\sklearn\metrics\classification.py:1439: UndefinedMetricWarning: Recall and F-score are ill-defined and being set to 0.0 in labels with no true samples.
## 'recall', 'true', average, warn_for)gb=GradientBoostingClassifier(n_estimators=150,max_depth=11)
start=time.time()
gb_model=gb.fit(count_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=gb_model.predict(count_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 43 cupping benefits cupping benefits
## 53 massage gun benefits massage gun benefits
## 85 ER ER
## 25 mental health services benefits mental health services benefits
## 55 cold stone benefits cold stone benefits
## 9 massage benefits massage benefits
## 24 physical therapy benefits physical therapy benefits
## 12 massage benefits massage benefits
## 28 mental health services benefits mental health services benefits
## 56 cold stone benefits cold stone benefits
## 40 cupping benefits cupping benefits
## 75 Not Professional Not Professional
## 19 Not Professional physical therapy benefits
## 72 Not Professional Not Professional
## 26 mental health services benefits mental health services benefits
## 61 cold stone benefits cold stone benefits
## 14 massage benefits massage benefits
## 10 massage benefits massage benefitsfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.9444444444444444print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[1 0 0 0 0 0 0 0]
## [0 2 0 0 0 0 0 0]
## [0 0 3 0 0 0 0 0]
## [0 0 0 2 0 0 0 0]
## [0 0 0 0 4 0 0 0]
## [0 0 0 0 0 1 0 0]
## [0 0 0 0 0 0 3 0]
## [0 1 0 0 0 0 0 1]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## ER 1.00 1.00 1.00 1
## Not Professional 0.67 1.00 0.80 2
## cold stone benefits 1.00 1.00 1.00 3
## cupping benefits 1.00 1.00 1.00 2
## massage benefits 1.00 1.00 1.00 4
## massage gun benefits 1.00 1.00 1.00 1
## mental health services benefits 1.00 1.00 1.00 3
## physical therapy benefits 1.00 0.50 0.67 2
##
## accuracy 0.94 18
## macro avg 0.96 0.94 0.93 18
## weighted avg 0.96 0.94 0.94 18TF-IDF RFC and GBC
stopwords = nltk.corpus.stopwords.words('english')
ps=nltk.PorterStemmer()
wn=nltk.WordNetLemmatizer()
data=pd.read_csv("benefitsContraindications3.csv", encoding='unicode_escape')
data.columns=['Document','Source','Topic','InternetSearch','Contraindications','RisksSideEffects']def count_punct(text):
count=sum([1 for char in text if char in string.punctuation])
return round(count/(len(text)-text.count(" ")),3)*100data['body_length']=data['Document'].apply(lambda x: len(x)-x.count(" "))
data['punct%']= data['Document'].apply(lambda x: count_punct(x))
def clean_text(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=[ps.stem(word) for word in tokens if word not in stopwords]#these are lists for the count vectorizer
return text
def lemmatize(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=[wn.lemmatize(word) for word in tokens if word not in stopwords]#these are lists for the count vectorizer
return text
data['Cleaned_text']=data['Document'].apply(lambda x: clean_text(x))
data['Lemmatized']=data['Document'].apply(lambda x: lemmatize(x))
data.head()## Document ... Lemmatized
## 0 Chiropractic adjustments and treatments serve ... ... [chiropractic, adjustment, treatment, serve, n...
## 1 \r\nUnitedHealthcare Combats Opioid Crisis wit... ... [, unitedhealthcare, combat, opioid, crisis, n...
## 2 The Safety of Chiropractic Adjustments\r\nBy ... ... [, safety, chiropractic, adjustment, lana, bar...
## 3 Advanced Chiropractic Relief: 8 Key Benefits o... ... [advanced, chiropractic, relief, 8, key, benef...
## 4 Heading to the spa can be a pampering treat, b... ... [heading, spa, pampering, treat, also, huge, b...
##
## [5 rows x 10 columns]
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(data[['Document','body_length','punct%','Cleaned_text','Lemmatized']],data['Topic'],test_size=0.20)
from sklearn.feature_extraction.text import TfidfVectorizer
tfidf_vect=TfidfVectorizer(analyzer=lemmatize)
tfidf_vect_fit=tfidf_vect.fit(X_train['Document'])
tfidf_train=tfidf_vect_fit.transform(X_train['Document'])
tfidf_test=tfidf_vect_fit.transform(X_test['Document'])len(tfidf_vect_fit.get_feature_names())## 4534tfidf_vect_fit.get_feature_names()[200:350]## ['activation', 'active', 'actively', 'activity', 'actual', 'actually', 'actuallyyou', 'acupoints', 'acupressure', 'acupuncture', 'acupuncturist', 'acupuncturists', 'acute', 'ad', 'add', 'added', 'addiction', 'adding', 'addition', 'additional', 'address', 'addressed', 'addressing', 'addthis', 'adeline', 'adequate', 'adhesion', 'adjunct', 'adjust', 'adjustable', 'adjusted', 'adjusting', 'adjustment', 'administered', 'administering', 'administration', 'adminmcb', 'admission', 'admitted', 'adolescent', 'adopted', 'adopting', 'adore', 'adult', 'adults1', 'advanced', 'advantage', 'advantageous', 'adverse', 'adversity', 'advertisement', 'advice', 'advise', 'advisory', 'advocate', 'aerobic', 'affair', 'affect', 'affected', 'affecting', 'affiliate', 'affirm', 'affordable', 'afraid', 'afterhours', 'afterward', 'age', 'agency', 'aggression', 'aggressive', 'aggressively', 'agility', 'aging', 'ago', 'agree', 'ahead', 'aid', 'aiding', 'ailment', 'aim', 'air', 'airport', 'ajman', 'al', 'alchemist', 'alcohol', 'alcoholsoaked', 'alert', 'align', 'aligning', 'alignment', 'alike', 'allergic', 'allergy', 'alleviate', 'alleviates', 'alleviating', 'alleviation', 'alliance', 'allison', 'allow', 'allowing', 'allows', 'almost', 'alone', 'along', 'alongside', 'already', 'alright', 'also', 'alter', 'altered', 'alternate', 'alternating', 'alternative', 'alters', 'although', 'altogether', 'always', 'alzheimers', 'amateur', 'amazing', 'amazon', 'amazoncom', 'ambulance', 'ameliorating', 'america', 'american', 'among', 'amount', 'amplitude', 'amy', 'analgesic', 'analysis', 'analyzed', 'ancient', 'andor', 'anecdotal', 'anemia', 'angeles', 'anger', 'angry', 'anhedonia', 'animal', 'aniston', 'ankle', 'ann', 'annals', 'anne', 'annie']tfidf_train_vect=pd.concat([X_train[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(tfidf_train.toarray())],axis=1)
tfidf_test_vect=pd.concat([X_test[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(tfidf_test.toarray())],axis=1)tfidf_train_vect.head()## Document body_length ... 4532 4533
## 0 \r\nBenefits of a Physical Therapist Career\r\... 3006 ... 0.0 0.0
## 1 enefits of Cold Stone Massage\r\n\r\nOne of th... 2651 ... 0.0 0.0
## 2 \r\nCupping Therapy\r\nIn this Article\r\n\r\n... 1975 ... 0.0 0.0
## 3 ll You Need To Know About Massage Gun\r\n31 Au... 4598 ... 0.0 0.0
## 4 \r\nGetting Started with Cold Stone Massage Th... 3996 ... 0.0 0.0
##
## [5 rows x 4539 columns]from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import precision_recall_fscore_support as score
import time
np.random.seed(45678)rf=RandomForestClassifier(n_estimators=150, max_depth=None, n_jobs=-1)
start=time.time()
rf_model=rf.fit(tfidf_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=rf_model.predict(tfidf_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 55 massage benefits cold stone benefits
## 71 Not Professional Not Professional
## 59 cold stone benefits cold stone benefits
## 80 Not Professional Not Professional
## 17 physical therapy benefits physical therapy benefits
## 33 Not Professional chiropractic benefits
## 32 chiropractic benefits chiropractic benefits
## 6 massage benefits massage benefits
## 11 massage benefits massage benefits
## 9 massage benefits massage benefits
## 79 Not Professional Not Professional
## 61 cold stone benefits cold stone benefits
## 54 massage gun benefits massage gun benefits
## 18 physical therapy benefits physical therapy benefits
## 24 physical therapy benefits physical therapy benefits
## 16 physical therapy benefits physical therapy benefits
## 10 massage benefits massage benefits
## 0 massage benefits chiropractic benefitsfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.8333333333333334print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[3 0 0 0 0 0]
## [1 1 0 1 0 0]
## [0 0 2 1 0 0]
## [0 0 0 4 0 0]
## [0 0 0 0 1 0]
## [0 0 0 0 0 4]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 0.75 1.00 0.86 3
## chiropractic benefits 1.00 0.33 0.50 3
## cold stone benefits 1.00 0.67 0.80 3
## massage benefits 0.67 1.00 0.80 4
## massage gun benefits 1.00 1.00 1.00 1
## physical therapy benefits 1.00 1.00 1.00 4
##
## accuracy 0.83 18
## macro avg 0.90 0.83 0.83 18
## weighted avg 0.88 0.83 0.82 18gb=GradientBoostingClassifier(n_estimators=150,max_depth=11)
start=time.time()
gb_model=gb.fit(tfidf_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=gb_model.predict(tfidf_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 55 Not Professional cold stone benefits
## 71 Not Professional Not Professional
## 59 cold stone benefits cold stone benefits
## 80 Not Professional Not Professional
## 17 physical therapy benefits physical therapy benefits
## 33 chiropractic benefits chiropractic benefits
## 32 chiropractic benefits chiropractic benefits
## 6 Not Professional massage benefits
## 11 massage benefits massage benefits
## 9 massage benefits massage benefits
## 79 massage benefits Not Professional
## 61 cold stone benefits cold stone benefits
## 54 massage gun benefits massage gun benefits
## 18 physical therapy benefits physical therapy benefits
## 24 physical therapy benefits physical therapy benefits
## 16 physical therapy benefits physical therapy benefits
## 10 massage benefits massage benefits
## 0 chiropractic benefits chiropractic benefitsfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.8333333333333334print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[2 0 0 1 0 0]
## [0 3 0 0 0 0]
## [1 0 2 0 0 0]
## [1 0 0 3 0 0]
## [0 0 0 0 1 0]
## [0 0 0 0 0 4]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 0.50 0.67 0.57 3
## chiropractic benefits 1.00 1.00 1.00 3
## cold stone benefits 1.00 0.67 0.80 3
## massage benefits 0.75 0.75 0.75 4
## massage gun benefits 1.00 1.00 1.00 1
## physical therapy benefits 1.00 1.00 1.00 4
##
## accuracy 0.83 18
## macro avg 0.88 0.85 0.85 18
## weighted avg 0.86 0.83 0.84 18N-Grams Vectorization for RFC and GBC
stopwords = nltk.corpus.stopwords.words('english')
ps=nltk.PorterStemmer()
wn=nltk.WordNetLemmatizer()
data=pd.read_csv("benefitsContraindications3.csv", encoding='unicode_escape')
data.columns=['Document','Source','Topic','InternetSearch','Contraindications','RisksSideEffects']def count_punct(text):
count=sum([1 for char in text if char in string.punctuation])
return round(count/(len(text)-text.count(" ")),3)*100data['body_length']=data['Document'].apply(lambda x: len(x)-x.count(" "))
data['punct%']= data['Document'].apply(lambda x: count_punct(x))def clean_text(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+',text)
text=" ".join([ps.stem(word) for word in tokens if word not in stopwords])#unlisted with N-grams vectorization
return text
def lemmatize(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=" ".join([wn.lemmatize(word) for word in tokens if word not in stopwords])#unlisted with N-grams vectorization
#text=[wn.lemmatize(word) for word in tokens if word not in stopwords]#when using count Vectorization its a list
#or else single letters returned.
return text data['Cleaned_text']=data['Document'].apply(lambda x: clean_text(x))
data['Lemmatized']=data['Document'].apply(lambda x: lemmatize(x))
data.head()## Document ... Lemmatized
## 0 Chiropractic adjustments and treatments serve ... ... chiropractic adjustment treatment serve need m...
## 1 \r\nUnitedHealthcare Combats Opioid Crisis wit... ... unitedhealthcare combat opioid crisis nonopio...
## 2 The Safety of Chiropractic Adjustments\r\nBy ... ... safety chiropractic adjustment lana barhum me...
## 3 Advanced Chiropractic Relief: 8 Key Benefits o... ... advanced chiropractic relief 8 key benefit chi...
## 4 Heading to the spa can be a pampering treat, b... ... heading spa pampering treat also huge boost he...
##
## [5 rows x 10 columns]from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(data[['Document','body_length','punct%','Cleaned_text','Lemmatized']],data['Topic'],test_size=0.20)from sklearn.feature_extraction.text import CountVectorizern_gram_vect=CountVectorizer(ngram_range=(1,4))type(X_train['Cleaned_text'])## <class 'pandas.core.series.Series'>X_train['Cleaned_text'].head()## 69 avail good massag plu servic sex top nice romant
## 68 good morn ajman spend day beauti girl monday r...
## 56 cold stone massag benefici adminmcb januari 21...
## 0 chiropract adjust treatment serv need million ...
## 31 21 benefit chiropract adjust woman chiropract...
## Name: Cleaned_text, dtype: objectX_train['Lemmatized'].head()## 69 available good massage plus service sex top ni...
## 68 good morning ajman spend day beautiful girl mo...
## 56 cold stone massage beneficial adminmcb january...
## 0 chiropractic adjustment treatment serve need m...
## 31 21 benefit chiropractic adjustment woman chir...
## Name: Lemmatized, dtype: objectn_gram_vect_fit=n_gram_vect.fit(X_train['Lemmatized'])
n_gram_train=n_gram_vect_fit.transform(X_train['Lemmatized'])
n_gram_test=n_gram_vect_fit.transform(X_test['Lemmatized'])len(n_gram_vect_fit.get_feature_names())## 73294print(n_gram_vect_fit.get_feature_names()[200:500])## ['18999 amazoncom damkees professional', '18999 damkee', '18999 damkee currently', '18999 damkee currently sale', '18h', '18h lmassagebodytobody', '18h lmassagebodytobody massage', '18h lmassagebodytobody massage massage', '19', '19 percent', '19 percent spinal', '19 percent spinal imaging', '192', '192 people', '192 people backrelated', '192 people backrelated pain', '1950s', '1950s across', '1950s across hospital', '1950s across hospital china', '1950s cupping', '1950s cupping also', '1950s cupping also practiced', '1996', '1996 study', '1996 study rand', '1996 study rand corporation', '19999', '19999 amazoncom', '19999 amazoncom 4000', '19999 amazoncom 4000 amazon', '20', '20 2015', '20 2015 therapy', '20 2015 therapy comment', '20 different', '20 different adjustable', '20 different adjustable speed', '20 minute', '20 minute depending', '20 minute depending nature', '20 minute long', '20 minute long say', '20 year', '20 year without', '20 year without adverse', '2000', '2000 3000', '2000 3000 hour', '2000 3000 hour supervised', '2001', '2001 overview', '2001 overview scoring', '2001 overview scoring phq9', '2001 study', '2001 study published', '2001 study published canadian', '2003', '2003 evaluated', '2003 evaluated 183', '2003 evaluated 183 patient', '2003 scored', '2003 scored might', '2003 scored might proceed', '2005', '2005 study', '2005 study published', '2005 study published international', '2007', '2007 according', '2007 according national', '2007 according national health', '2007 article', '2007 article chicago', '2007 article chicago tribune', '2007 likely', '2007 likely seek', '2007 likely seek coaching', '2007 stress', '2007 stress also', '2007 stress also lead', '2008', '2008 center', '2008 center disease', '2008 center disease control', '2008 grown', '2008 grown become', '2008 grown become favorite', '2008 researcher', '2008 researcher evaluated', '2008 researcher evaluated chiropractic', '2009', '2009 disease', '2009 disease link', '2009 disease link inflammation', '2009 found', '2009 found reflective', '2009 found reflective journaling', '2009 reduction', '2009 reduction chronic', '2009 reduction chronic pain', '2010', '2010 study', '2010 study published', '2010 study published journal', '2011', '2011 cnnmoneycom', '2011 cnnmoneycom gave', '2011 cnnmoneycom gave physical', '2011 number', '2011 number adult', '2011 number adult seek', '2011 review', '2011 review 26', '2011 review 26 clinical', '2011 study', '2011 study published', '2011 study published annals', '2012', '2012 report', '2012 report published', '2012 report published plo', '2012 review', '2012 review studiestrusted', '2012 review studiestrusted source', '2012 well', '2012 well benefit', '2012 well benefit society', '2013', '2013 2011', '2013 2011 cnnmoneycom', '2013 2011 cnnmoneycom gave', '2013 illustrating', '2013 illustrating benefit', '2013 illustrating benefit journaling', '2014', '2014 2024a', '2014 2024a much', '2014 2024a much quicker', '2014 study', '2014 study suggested', '2014 study suggested vibration', '2015', '2015 journal', '2015 journal traditional', '2015 journal traditional complementary', '2015 report', '2015 report published', '2015 report published journal', '2015 review', '2015 review evidence', '2015 review evidence found', '2015 therapy', '2015 therapy comment', '2015 therapy comment physical', '2015patricia', '2015patricia mayrhofer', '2015patricia mayrhofer spatechnique', '2015patricia mayrhofer spatechnique articles2', '2016', '2016 930', '2016 930 robert', '2016 930 robert shmerling', '2016 cold', '2016 cold stone', '2016 cold stone massage', '2016 michael', '2016 michael phelps', '2016 michael phelps permanently', '2016 mr', '2016 mr castaneda', '2016 mr castaneda written', '2016 study', '2016 study published', '2016 study published journal', '2016 summer', '2016 summer olympics', '2016 summer olympics share', '2016 used', '2016 used cupping', '2016 used cupping easily', '2017', '2017 2018', '2017 2018 cdc', '2017 2018 cdc report', '2017 2018 legal', '2017 2018 legal battle', '2017 chiropractor', '2017 chiropractor tout', '2017 chiropractor tout treatment', '2017 nba', '2017 nba final', '2017 nba final irving', '2017 study', '2017 study found', '2017 study found structure', '2018', '2018 84000', '2018 84000 topic', '2018 84000 topic physical', '2018 according', '2018 according american', '2018 according american massage', '2018 cdc', '2018 cdc report', '2018 cdc report physical', '2018 finding', '2018 finding help', '2018 finding help comment', '2018 galluppalmer', '2018 galluppalmer college', '2018 galluppalmer college chiropractic', '2018 getty', '2018 getty image', '2018 getty image cupping', '2018 individual', '2018 individual salary', '2018 individual salary vary', '2018 largest', '2018 largest percentage', '2018 largest percentage working', '2018 legal', '2018 legal battle', '2018 legal battle aim', '2018 study', '2018 study led', '2018 study led dr', '2019', '2019 1011', '2019 1011 article', '2019 1011 article based', '2019 1122', '2019 1122 share', '2019 1122 share article', '2019 342', '2019 342 pm', '2019 342 pm edt', '2019 early', '2019 early 2020', '2019 early 2020 many', '2019 found', '2019 found patient', '2019 found patient high', '2019 massage', '2019 massage gun', '2019 massage gun one', '2019 physical', '2019 physical therapist', '2019 physical therapist stretch', '2019 puhhhagetty', '2019 puhhhagetty image', '2019 puhhhagetty image come', '2019 rich', '2019 rich barnesgetty', '2019 rich barnesgetty image', '2019 story', '2019 story previously', '2019 story previously published', '2019 thai', '2019 thai massage', '2019 thai massage full', '2019 unitedhealthcare', '2019 unitedhealthcare uhc', '2019 unitedhealthcare uhc combatting', '2020', '2020 2021', '2020 2021 end', '2020 2021 end expansion', '2020 beyond', '2020 beyond people', '2020 beyond people say', '2020 click', '2020 click read', '2020 click read user', '2020 depressed', '2020 depressed man', '2020 depressed man lying', '2020 karen', '2020 karen green', '2020 karen green last', '2020 many', '2020 many people', '2020 many people started', '2020 may', '2020 may look', '2020 may look like', '2020 orthopedics', '2020 orthopedics orthopedics', '2020 orthopedics orthopedics home', '2021', '2021 end', '2021 end expansion', '2021 end expansion period', '2021 opioid', '2021 opioid use', '2021 opioid use decrease', '2024', '2024 love', '2024 love job', '2024 love job helping']n_gram_train_df=pd.concat([X_train[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(n_gram_train.toarray())],axis=1)
n_gram_test_df=pd.concat([X_test[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(n_gram_test.toarray())],axis=1)n_gram_train_df.head()## Document body_length ... 73292 73293
## 0 Available now.. good massage plus service sex.... 68 ... 0 0
## 1 Good morning Ajman.\r\nSpend your day with bea... 157 ... 0 0
## 2 Cold Stone Massage is Beneficial, Too - by adm... 1606 ... 0 0
## 3 Chiropractic adjustments and treatments serve ... 2497 ... 0 0
## 4 \r\n21 Benefits of Chiropractic Adjustments\r\... 1546 ... 0 0
##
## [5 rows x 73299 columns]from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import precision_recall_fscore_support as score
import timerf=RandomForestClassifier(n_estimators=150, max_depth=None, n_jobs=-1)
start=time.time()
rf_model=rf.fit(n_gram_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=rf_model.predict(n_gram_test)
end=time.time()
pred_time=(end-start)
prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 3 massage benefits chiropractic benefits
## 41 cupping benefits cupping benefits
## 62 Not Professional cold stone benefits
## 48 Not Professional massage gun benefits
## 43 Not Professional cupping benefits
## 70 Not Professional Not Professional
## 25 Not Professional mental health services benefits
## 75 Not Professional Not Professional
## 21 physical therapy benefits physical therapy benefits
## 15 Not Professional massage benefits
## 77 Not Professional Not Professional
## 55 massage benefits cold stone benefits
## 11 Not Professional massage benefits
## 36 Not Professional cupping benefits
## 33 Not Professional chiropractic benefits
## 32 Not Professional chiropractic benefits
## 18 Not Professional physical therapy benefits
## 84 Not Professional ERfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.2777777777777778print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[0 1 0 0 0 0 0 0 0]
## [0 3 0 0 0 0 0 0 0]
## [0 2 0 0 0 1 0 0 0]
## [0 1 0 0 0 1 0 0 0]
## [0 2 0 0 1 0 0 0 0]
## [0 2 0 0 0 0 0 0 0]
## [0 1 0 0 0 0 0 0 0]
## [0 1 0 0 0 0 0 0 0]
## [0 1 0 0 0 0 0 0 1]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## ER 0.00 0.00 0.00 1
## Not Professional 0.21 1.00 0.35 3
## chiropractic benefits 0.00 0.00 0.00 3
## cold stone benefits 0.00 0.00 0.00 2
## cupping benefits 1.00 0.33 0.50 3
## massage benefits 0.00 0.00 0.00 2
## massage gun benefits 0.00 0.00 0.00 1
## mental health services benefits 0.00 0.00 0.00 1
## physical therapy benefits 1.00 0.50 0.67 2
##
## accuracy 0.28 18
## macro avg 0.25 0.20 0.17 18
## weighted avg 0.31 0.28 0.22 18
##
##
## C:\Users\m\Anaconda2\envs\python36\lib\site-packages\sklearn\metrics\classification.py:1437: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.
## 'precision', 'predicted', average, warn_for)gb=GradientBoostingClassifier(n_estimators=150,max_depth=11)
start=time.time()
gb_model=gb.fit(n_gram_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=gb_model.predict(n_gram_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 3 chiropractic benefits chiropractic benefits
## 41 cupping benefits cupping benefits
## 62 cold stone benefits cold stone benefits
## 48 massage gun benefits massage gun benefits
## 43 cupping benefits cupping benefits
## 70 Not Professional Not Professional
## 25 mental health services benefits mental health services benefits
## 75 Not Professional Not Professional
## 21 physical therapy benefits physical therapy benefits
## 15 massage benefits massage benefits
## 77 Not Professional Not Professional
## 55 massage benefits cold stone benefits
## 11 massage benefits massage benefits
## 36 cupping benefits cupping benefits
## 33 chiropractic benefits chiropractic benefits
## 32 chiropractic benefits chiropractic benefits
## 18 physical therapy benefits physical therapy benefits
## 84 ER ERfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.9444444444444444print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[1 0 0 0 0 0 0 0 0]
## [0 3 0 0 0 0 0 0 0]
## [0 0 3 0 0 0 0 0 0]
## [0 0 0 1 0 1 0 0 0]
## [0 0 0 0 3 0 0 0 0]
## [0 0 0 0 0 2 0 0 0]
## [0 0 0 0 0 0 1 0 0]
## [0 0 0 0 0 0 0 1 0]
## [0 0 0 0 0 0 0 0 2]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## ER 1.00 1.00 1.00 1
## Not Professional 1.00 1.00 1.00 3
## chiropractic benefits 1.00 1.00 1.00 3
## cold stone benefits 1.00 0.50 0.67 2
## cupping benefits 1.00 1.00 1.00 3
## massage benefits 0.67 1.00 0.80 2
## massage gun benefits 1.00 1.00 1.00 1
## mental health services benefits 1.00 1.00 1.00 1
## physical therapy benefits 1.00 1.00 1.00 2
##
## accuracy 0.94 18
## macro avg 0.96 0.94 0.94 18
## weighted avg 0.96 0.94 0.94 18Second part: Lemmatized Tokens & 85/15 Train/Test split & RFC | GBC
Count Vectorizer RFC and GBC
stopwords = nltk.corpus.stopwords.words('english')
ps=nltk.PorterStemmer()
wn=nltk.WordNetLemmatizer()
data=pd.read_csv("benefitsContraindications3.csv", encoding='unicode_escape')
data.columns=['Document','Source','Topic','InternetSearch','Contraindications','RisksSideEffects']data <- read.csv('benefitsContraindications3.csv',sep=',',header=TRUE, na.strings=c('',' ','NA'))
colnames(data)## [1] "Document" "Source" "Topic"
## [4] "InternetSearch" "Contraindications" "risksAdverseEffects"head(data,5)## Document
## 1 Chiropractic adjustments and treatments serve the needs of millions of people around the world.\n\n \n\nAdjustments offer effective, non-invasive and cost-effective solutions to neck and back pain, as well as a myriad of other medical issues.\n\nHave you ever stopped to wonder how many of us suffer from neck and back stiffness or pain?\n\nApart from the obvious discomfort, simple daily tasks such as driving a car, crossing a busy street and picking things up from the floor can become all too challenging for individuals experiencing such pain.\n\nAs anyone who has experienced pain would know, having restricted movement can be debilitating and unfortunately, our busy world doesn?t allow for us to stop.\n\n \nSome of the benefits of long-term chiropractic care include:\n\n Chiropractors can identify mechanical issues that cause spine-related pain and offer a series of adjustments that provide near immediate relief. Following appointments, patients often report feeling their symptoms noticeably better.\n When a chiropractor performs an adjustment, they can help restore movement in joints that have ?locked up?. This becomes possible as treatment allows muscles surrounding joints to relax, thereby reducing joint stiffness.\n Many factors affect health, including exercise patterns, nutrition, sleep, heredity and the environment in which we live. Rather than just treat symptoms of the disease, chiropractic care focuses on a holistic approach to naturally maintain health and resist disease.\n Chiropractic adjustments help restore normal function and movement to the entire body. Many patients report an improvement in their ability to move with efficiency and strength.\n Many patients find delight in the results chiropractic adjustments have on old and chronic injuries. Whether an injury is, in fact, new or old, chiropractic care can help reduce pain, restore mobility and provide quick pain relief to all joints in the body. Such care can help maintain better overall health and thus faster recovery time.\n\nHave you ever noticed that when you are in pain and unable to perform regular or favorite activities, it can put a strain on emotional and mental well-being?\n\nFor example, the increased stress from not being able to properly perform a paid job. This, in turn, can have a negative impact on physical health with increases in heart rate and blood pressure. The domino effect often continues with sleep becoming disturbed, with resulting lethargy and tiredness during the day. Does anyone really feel up to exercising in this state?\n\nChiropractic care is a natural method of healing the body?s communication system and never relies on the use of pharmaceutical drugs or invasive surgery.\n\nDoctors of Chiropractic work collaboratively with other healthcare professionals. Should your condition require the attention of another healthcare profession, that recommendation or referral will be made.\n\n
## 2 \nUnitedHealthcare Combats Opioid Crisis with Non-Opioid Benefits\nPhysical therapy and chiropractic care can prevent or reduce expensive, invasive spinal procedures, such as imaging or surgery, to reduce opioid use and cut costs.\nUnitedHealthcare, opioid, physical therapy, healthcare spending\n\nSource: Thinkstock\nShare on Twitter\n\nBy Kelsey Waddill\n\nOctober 29, 2019 - UnitedHealthcare (UHC) is combatting the opioid epidemic and high healthcare costs with new physical therapy and chiropractic care benefits to prevent, delay, or in some cases substitute for invasive spinal procedures.\n\n?With millions of Americans experiencing low back pain currently or at some point during their lifetimes, we believe this benefit design will help make a meaningful difference by improving health outcomes while reducing costs,? said Anne Docimo, MD, UnitedHealthcare chief medical officer.\n\nLower back pain is in part responsible for sustaining the opioid epidemic and also increases healthcare costs.\nDig Deeper\n\n How Major Payers Provide Substance Abuse Care for Opioid Misuse\n Opioid Overdoses Fall by 2% from 2017 to 2018, CDC Reports\n Physical Therapy, Chiropractic Back Care Cut Opioid Use, Costs\n\nAlthough opioid overdoses fell by two percent from 2017 to 2018 and a legal battles aim to hold pharmaceutical companies accountable, there is no end in sight for the opioid epidemic. Industry professionals are still grappling with the balance between cutting opioid prescriptions will working to reduce patient pain.\n\nCommon conditions such as low back pain bolster the epidemic?s presence, with clinicians still prescribing the opioids against best practice recommendations. According to a recent OptumLabs study, 9 percent of patients with newly diagnosed low back pain are prescribed opioids and lower back pain currently contributes 52 percent to the overall opioid prescription rate.\n\nIn addition to boosting opioids distribution, alternative, invasive lower back pain treatments can significantly impact healthcare spending.\n\nIt is not new information that physical therapy and chiropractic care are effective, lower cost alternatives to spinal imaging or surgery. However, payers are still in the process of adopting the method.\n\nTo counteract the high-cost, high-risk potential of using opioids to treat back pain, UHC created a benefit that does not rely on medication or technology but rather on physical therapy and chiropractic care.\n\nThe benefit allows eligible employers to offer physical therapist and chiropractor visits with no out-of-pocket costs. Members who already receive physical therapist and chiropractic care benefits under UHC?s employer-sponsored health plans and who have maxed out their visits will not receive additional visits under this benefit.\n\nHowever, for those who still have visits to use and who choose physical therapy or chiropractic care over other forms of treatment, the copay or deductible for those visits will be waived and they will receive three visits at no cost.\n\nUHC has high expectations for the fiscal and physical impacts of this benefit.\n\nAccording to UHC?s analysis, the health payer expects that by 2021, opioid use will decrease by 19 percent. Spinal imaging test frequency and spinal surgeries will be reduced by 22 percent and 21 percent, respectively. In addition to these specific goals, UHC hopes to see a decrease in the overall cost of spinal care.\n\nThe same OptumLabs study demonstrated that UHC?s expectations are not without precedent.\n\nThe study looked at the correlation between out-of-pocket costs and patient utilization of noninvasive treatments. Researchers discovered that members whose copay was over $30 were a little under 30 percent less likely to choose physical therapy as opposed to more invasive treatments.\n\nAn American Journal of Managed Care study in June 2019 found that patients with high deductibles, typically over $1,000, were less likely to visit physical therapy.\n\nEligible employers may be brand new or renewing their membership. They must be fully insured and over 51 or more employees strong. The benefit is currently available in Connecticut, Florida, Georgia, New York, and North Carolina.\n\nHowever, UHC plans to expand the benefit from 2020 into 2021. By the end of this expansion period, the benefit will also be available to self-funded employers and organizations with an employee population between 2 and 50. The benefit will span ten states, primarily in the southeast.\n\n?This new benefit design may help encourage people with low back pain to get the right care at the right time and in the right setting, helping expand access to evidence-based and more affordable treatments,? said Docimo.
## 3 The Safety of Chiropractic Adjustments\nBy Lana Barhum\nMedically reviewed by Richard N. Fogoros, MD\nUpdated on January 31, 2020\nOrthopedics\nMore in Orthopedics\n\n Home Office Ergonomics\n Sprains & Strains\n Fractures & Broken Bones\n Physical Therapy\n Orthopedic Surgery\n Osteoporosis\n Pediatric Orthopedics\n Sports Injuries\n\nView All\nIn This Article\n\n Chiropractic Adjustment\n What Research Shows\n Safety\n\nChiropractic adjustment, also called spinal manipulation, is a procedure done by a chiropractor using the hands or small instruments to apply controlled force to a spinal joint. The goal is to improve spinal motion and physical function of the entire body. Chiropractic adjustment is safe when performed by someone who is properly trained and licensed to practice chiropractic care. Complications are rare, but they are possible. Learn more about both the benefits and risks.\nChiropractic adjustment\nVerywell / Brianna Gilmartin \nChiropractic Adjustment\n\nOne of the most important reasons people seek chiropractic care is because it is a completely drug-free therapy. Someone dealing with joint pain, back pain, or headaches might consider visiting a chiropractor.\n\nThe goal of chiropractic adjustment is to place the body into a proper position so the body can heal itself. Treatments are believed to reduce stress on the immune system, reducing the potential for disease. Chiropractic care aims to address the entire body, including a person?s ability to move, perform, and even think.\nWhat Research Shows\n\nMany people wonder how helpful chiropractic care is in treating years of trauma and poor posture. There have been numerous studies showing the therapeutic benefits of chiropractic care.\nSciatica\n\nSciatica is a type of pain affecting the sciatic nerve, the large nerve extending from the low back down the back of the legs. Other natural therapies don?t always offer relief and most people want to avoid steroid injections and surgery, so they turn to chiropractic care.\n\nA double-blind trial reported in the Spine Journal compared active and simulated chiropractic manipulations in people with sciatic nerve pain. Active manipulations involved the patient laying down and receiving treatment from a chiropractor. Stimulated manipulations involved electrical muscle stimulation with electrodes placed on the skin to send electrical pulses to different parts of the body.\n\nThe researchers determined active manipulation offered more benefits than stimulated. The people who received active manipulations experienced fewer days of moderate or severe pain and other sciatica symptoms. They also reported no adverse effects.\nNeck Pain\n\nOne study reported in the Annals of Internal Medicine looked at different therapies for treating neck pain. They divided 272 study participants into three groups: one that received spinal manipulation from a chiropractic doctor, a second group given over-the-counter (OTC) pain relievers, narcotics, and muscle relaxers, and a third group who did at-home exercises. \n\nAfter 12 weeks, patients reported a 75% pain reduction, with the chiropractic treatment group achieving the most improvement. About 57% of the chiropractic group achieved pain reduction, while 48% received pain reduction from exercising, and 33% from medication.\n\nAfter one year, 53% of the drug-free groups continued to report pain relief compared to only 38% of those taking pain medications. \nHeadaches\n\nCervicogenic headaches and migraines are commonly treated by chiropractors. Cervicogenic headaches are often called secondary headaches because pain is usually referred from another source, usually the neck. Migraine headaches cause severe, throbbing pain and are generally experienced on one side of the head. There are few non-medicinal options for managing both types of chronic headaches.\n\nResearch reported in the Journal of Manipulative and Physiological Therapeutics suggests chiropractic care, specifically spinal manipulation, can improve migraines and cervicogenic headaches. \nFrozen Shoulder\n\nFrozen shoulder affects the shoulder joint and involves pain and stiffness that develops gradually and gets worse. Frozen shoulder can be quite painful, and treatment involves preserving as much range of motion in the shoulder as possible and managing pain.\n\nA clinical trial reported in the Journal of Chiropractic Medicine described how patients suffering from frozen shoulder responded to chiropractic treatment. Of the 50 patients, 16 completely recovered, 25 showed a 75 to 90% improvement, and eight showed a 50 to 75% improvement. Only one person showed zero to 50% improvement. The researchers concluded most people can get improvement by treating frozen shoulder with chiropractic treatment.\nPreventing Need for Surgery\n\nChiropractic care may reduce the need for back surgery. Guidelines reported in the Journal of the American Medical Association suggest that it's reasonable for people suffering from back pain to try spinal manipulation before deciding on surgical intervention.\nLow Back Pain\n\nStudies have shown chiropractic care, including spinal manipulation, can provide relief from mild to moderate low back pain. In fact, spinal manipulation may work as well as other standard treatments, including pain-relief medications.\n\nA 2011 review of 26 clinical trials looked at the effectiveness of different treatments for chronic low back pain. What they found was that spinal manipulation is just as effective as other treatments for reducing back pain and improving function.\nSafety\n\n\nWhen chiropractors are correctly trained and licensed, chiropractic care is safe. Mild side effects are to be expected and include temporary soreness, stiffness, and tenderness in the treated area. However, you still want to do your research. Ask for a referral from your doctor. Look at the chiropractor?s website, including patient reviews. Meet with the chiropractor to discuss his or her treatment practices and ask about possible adverse effects related to treatment.\n\nIf you decide a chiropractor isn?t for you, consider seeing an osteopathic doctor. Osteopaths are fully licensed doctors who can practice all areas of medicine. They have received special training on the musculoskeletal system, which includes manual readjustments, myofascial release and other physical manipulation of bones and muscle tissues.
## 4 Advanced Chiropractic Relief: 8 Key Benefits of Chiropractor Care\n\nAre you one of the 50 million Americans who suffer from chronic pain? If so you?re probably intimately familiar with the feeling of pure desperation that can arise from an inability to find relief.\n\nIn addition to physical issues, chronic pain can cause anxiety, depression, and more. However, there could be a light at the end of the tunnel. Many people are finding advanced chiropractic relief that is completely changing their lives.\n\nYour body is a world in itself. At this very moment, more than a million chemical reactions are taking place in your body. It manufactures energy, it regulates your heartbeat, your breathing and it regenerates and heals itself. Everything takes place without your conscious knowledge, without you controlling it voluntarily. The master system that controls it all is your nervous system.\n\nThe nervous system is made out of your brain, spinal cord and all your nerves.\n\nThe energy that flows through your nervous system in your body is like electricity. In order to have that electric flow normally and freely, we need to have a well functioning spine. Whenever you have disruption of that flow, disease happens. That would be the case when your spine is misaligned or is not moving properly.\n\nDid you know that 90% of stimulation and nutrition to the brain is generated by the movement of the spine?\n\nThe more mechanically distorted a person is, the less energy is available for thinking, metabolism and healing.\n\nThis is why it is so important to have a healthy spine, a proper posture, to exercise, to eat properly ? all of it truly matters for your quality of life.\nChiropractors localize the areas of your spine that do not move properly ? referred to as vertebral subluxations ? and adjust them with a specific high speed, but yet gentle, thrust to improve spinal motion.\n\nWant to learn about some of the ways chiropractic care can help you? Keep reading for insight into some of the key benefits of seeing a chiropractor.\n\nThe benefits of chiropractic care are numerous:\n\n1. Lower Blood Pressure\n\nStudies show that chiropractic treatment can lower your blood pressure. Sometimes, this works just as well as a prescription blood pressure medication! This benefit can also last for as long as six months after treatment.\n\nHigh blood pressure can cause an array of serious side effects like nausea, fatigue, dizziness, and anxiety. Sufferers who haven?t found relief should consider consulting with a chiropractor. A chiropractic adjustment may be the solution.\n\nSome studies have shown that chiropractic adjustments can also help patients who are suffering from low blood pressure.\n\n2. Reduced Inflammation\n\nIn many cases, joint issues, pain, and tension are caused by inflammation in the body. Chiropractic adjustments can reduce inflammation.\n\nThis leads to relief of muscle tension, chronic back pain, and joint pain. These adjustments can sometimes also slow the progression of inflammation-related diseases, like arthritis.\n\n3. Better Sleep\n\nPatients who receive chiropractic adjustments report a significant improvement in their sleep patterns. If you regularly suffer from insomnia, visiting a chiropractor regularly may help. Also, when you experience pain relief, this will help you get a restful night?s sleep.\n\n4. Digestive Relief\n\nChiropractors often give nutritional advice as part of their services. However, this isn?t the only way that they provide patients with digestive relief.\n\nAdjusting the thoraco-lumbar spine restores the neurological function of your digestive system. Regular adjustments can help with chronic digestive issues.\n\n5. Stress Release\n\nEveryday life can cause muscle cramping, inflammation, and more. When you?re sore from working at a computer, heavy lifting, or just dealing with emotional stress, a chiropractic adjustment can help. This leads to greater comfort and advanced pain relief.\n\n6. Improvement of Neurological Conditions\n\nA chiropractic adjustment can also increase blood flow to the brain and increase the flow of cerebral spinal fluid. This means that patients suffering from neurological conditions like epilepsy and multiple sclerosis can significantly benefit from regular adjustments.\n\nThis is a relatively new area of study, but the potential is huge. Those suffering from these conditions will want to do some research. It?s important to find the best chiropractor in their area with experience dealing with these specific types of cases.\n\n7. Chiropractic care can improve communication from your brain to your muscles\n\nResearch seems to show that chiropractic care can improve your brain-body communication, helping your brain to be more aware of what is going on in the body so it can control your body better.\n\nBetter health, more energy and vitality are some of the positive effects of getting your spine adjusted. It sets your vertebrae back into motion freeing up the energy that travels through your nerves.\n\nChiropractic care is a partnership. The results patients want is a combination of what the chiropractor does and what the patient does.\n\nThere are many good things that can be changed and improved for a better lifestyle: exercise, good nutrition, good mental attitude and spinal adjustments.\n\nYour whole body will work better by having your nervous system free of interference. That is the essence of chiropractic care and is designed for you and your family.\n\n8. Pain Relief\n\nPerhaps the most well-known benefit of going to a chiropractor is pain relief. Adjustments can help with a huge array of painful conditions including the following.\n\nNeck and Lower Back Pain\n\nAdjustments are the most effective non-invasive pain relief method for this type of pain. They may help patients avoid having to take prescription pain management drugs.\n\nSciatica\n\nTreatments help relieve pressure on the nerve. This results in less severe pain that lasts for a fewer number of days.\n\nHeadaches\n\nChiropractic adjustments help headaches and migraines. They do this by treating back misalignment, muscle tension, and stress. Cervical spine manipulation was associated with significant improvement in headache outcomes in trials involving patients with neck pain and/or neck dysfunction and headache.\n\nChronic headaches can result from the abnormal positioning of the head and can be worsened from neck pressure and movement. Chiropractic removes the interference whether it may be from the distant muscle tightness in the back causing strain on your spine or an abnormal lordotic cervical curve and moving vertebrae.\nChiropractic care can reduce the duration of headaches, lower their intensity when they do occur and limit the frequency of their occurrence all together.\n\nMenstrual cramps\n\nChiropractic treatment removes tension from the pelvis and sacrum. It also regulates the neurological function communicating with the reproductive organs. Adjustments can also relieve the bloating, cramping, and pain associated with menstrual cramps\n\nAnyone who has tried traditional medical treatments and has been unable to find pain relief should experiment with chiropractic care. More often than not, you?ll be pleasantly surprised!\n\nBonus: Advanced Chiropractic Relief\n\nIn addition to the benefits listed above, adjustments can bring advanced chiropractic relief for a wide variety of other conditions as well as overall life improvement. A few examples include:\n\nScoliosis ? adjustments have shown to help with the pain, reduced range of motion, abnormal posture, and even difficulty breathing caused by this abnormal curvature of the spine\n\nVertigo ? an adjustment can help realign and balance the spine, thereby reducing the dizziness, nausea, and disorientation caused by vertigo\n\nSinus and allergy relief ? adjusting the upper cervical spine can help drain the sinuses and provide immediate and lasting relief from both long-term and seasonal allergies\n\nExpectant mothers ? women can experience relief from pain and morning sickness and are better able to maintain proper posture during and after pregnancy\n\nChildren?s issues ? treatments have been shown to help children with acid reflux, cholic, and ear infections\nAthletic performance ? the reduction in pain and inflammation is particularly beneficial for professional and amateur athletes\n\nStimulates the immune system ? chiropractic care helps to boost the immune system, speeding up the healing process following illnesses or injuries. One of the most important studies showing the positive effect chiropractic care can have on the immune system and general health was performed by Ronald Pero, Ph.D., chief of cancer prevention research at New York?s Preventive Medicine Institute and professor of medicine at New York University. Dr. Pero measured the immune systems of people under chiropractic care as compared to those in the general population and those with cancer and other serious diseases.\n\nIn his initial three-year study of 107 individuals who had been under chiropractic care for five years or more, the chiropractic patients were found to have a 200% greater immune competence than people who had not received chiropractic care, and 400% greater immune competence than people with cancer and other serious diseases. The immune system superiority of those under chiropractic care did not diminish with age.\n\nDr. Pero stated: ?When applied in a clinical framework, I have never seen a group other than this chiropractic group to experience a 200% increase over the normal patients. This is why it is so dramatically important. We have never seen such a positive improvement in a group.?\n\nAs you can see, there are almost limitless benefits to seeking chiropractic treatment. If you haven?t tried it yet, what are you waiting for?\n\nThere?s no need to accept pain and discomfort as a normal part of life. You have nothing to lose and everything to gain, so it only makes sense to find out more about this possibly life-changing approach to improving your health and wellness.
## 5 Heading to the spa can be a pampering treat, but it can also be a huge boost to your health and wellness! Massage therapy can relieve all sorts of ailments ? from physical pain, to stress and anxiety. People who choose to supplement their healthcare regimen with regular massages will not only enjoy a relaxing hour or two at the spa, but they will see the benefits carry through the days and weeks after the appointment!\n\n1\n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\nThese are the 10 most common benefits reported from massage therapy:\n\n1. Reduce Stress\n\nA relaxing day at the spa is a great way to unwind and de-stress. However, clients are sure to notice themselves feeling relaxed and at ease for days and even weeks after their appointments!\n\n \n\n2. Improve Circulation\n\nLoosening muscles and tendons allows increased blood flow throughout the body. Improving your circulation can have a number of positive effects on the rest of your body, including reduced fatigue and pain management!\n\n \n\n3. Reduce Pain\n\nMassage therapy is great for working out problem areas like lower back pain and chronic stiffness. A professional therapist will be able to accurately target the source of your pain and help achieve the perfect massage regimen.\n\n \n\n3\n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n4. Eliminate Toxins\n\nStimulating the soft tissues of your body will help to release toxins through your blood and lymphatic systems.\n\n \n\n5. Improve Flexibility\n\nMassage therapy will loosen and relax your muscles, helping your body to achieve its full range of movement potential.\n\n \n\n6. Improve Sleep\n\nA massage will encourage relaxation and boost your mood. Going to bed with relaxed and loosened muscles promotes more restful sleep, and you?ll feel less tired in the morning!\n\n \n\n7. Enhance Immunity\n\nStimulation of the lymph nodes re-charges the body?s natural defense system.\n\n \n\n8. Reduce Fatigue\n\nMassage therapy is known to boost mood and promote better quality sleep, thus making you feel more rested and less worn-out at the end of the day.\n\n \n\n2\n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n9. Alleviate Depression and Anxiety\n\nMassage therapy can help to release endorphins in your body, helping you to feel happy, energized, and at ease.\n\n \n\n10. Reduce post-surgery and post-injury swelling\n\nA professional massage is a great way to safely deal with a sports injury or post-surgery rehabilitation.\n\nDo you think that massage therapy could help you find relief in any of these areas? What improvements would you like to see in your health? Contact us today with your questions about massage therapy and see how we can help you get on the path to improved health and wellness!
## Source
## 1 https://coremedicalohio.com/benefits-of-long-term-chiropractic-care/?utm_source=ReviveOldPost&utm_medium=social&utm_campaign=ReviveOldPost
## 2 https://healthpayerintelligence.com/news/unitedhealthcare-combats-opioid-crisis-with-non-opioid-benefits
## 3 https://www.verywellhealth.com/is-chiropractic-adjustment-safe-4588279
## 4 https://hafkeychiropractic.com/advanced-chiropractic-relief-8-key-benefits-of-chiropractor-care/
## 5 https://www.urbannirvana.com/10-benefits-massage-therapy/
## Topic InternetSearch Contraindications
## 1 chiropractic benefits <NA> <NA>
## 2 chiropractic benefits <NA> <NA>
## 3 chiropractic benefits <NA> <NA>
## 4 chiropractic benefits <NA> <NA>
## 5 massage benefits google <NA>
## risksAdverseEffects
## 1 <NA>
## 2 <NA>
## 3 Risks and side effects associated with chiropractic adjustments may include:\n\n temporary headaches\n fatigue after treatment\n discomfort in parts of the body that were treated\n\nRare but serious risks associated with chiropractic adjustment include:\n\n stroke\n cauda equina syndrome, a condition involving pinched nerves in the lower part of the spinal canal\n worsening of herniated disks (although research isn't conclusive)\n\nIn addition to effectiveness, research has focused on the safety of chiropractic treatments, mainly spinal manipulation. \n\nOne 2017 review of 250 articles looked at serious adverse events and benign events associated with chiropractic care. Based on the evidence the researchers reviewed, serious adverse events accounted for one out of every two million spinal manipulations to 13 per 10,00 patients. Serious adverse events included spinal or neurological problems and cervical arterial strokes (dissection of any of the arteries in the neck).\n\nBenign events were more common and included more pain and higher levels of neck problems, but most were short-term problems.\n\nThe researchers confirmed serious adverse events were rare and often related to other preexisting conditions, while benign events are more common. However, the reasons for any types of adverse events are unknown.\n\nA second 2017 review looked 118 articles and found frequently described adverse events include stroke, headache and vertebral artery dissection (cervical arterial stroke). Forty-six percent of the reviews determined that spinal manipulation was safe, while 13% expressed concern of harm. The remaining studies were unclear or neutral. While the researchers did not offer an overall conclusion, they determined spinal manipulation can significantly be helpful, and some risk does exist.\nA Word From Verywell
## 4 <NA>
## 5 <NA>def count_punct(text):
count=sum([1 for char in text if char in string.punctuation])
return round(count/(len(text)-text.count(" ")),3)*100data['body_length']=data['Document'].apply(lambda x: len(x)-x.count(" "))
data['punct%']= data['Document'].apply(lambda x: count_punct(x))
def clean_text(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=[ps.stem(word) for word in tokens if word not in stopwords]#these are lists for the count vectorizer
return text
def lemmatize(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=[wn.lemmatize(word) for word in tokens if word not in stopwords]#these are lists for the count vectorizer
return text
data['Cleaned_text']=data['Document'].apply(lambda x: clean_text(x))
data['Lemmatized']=data['Document'].apply(lambda x: lemmatize(x))
data.head()## Document ... Lemmatized
## 0 Chiropractic adjustments and treatments serve ... ... [chiropractic, adjustment, treatment, serve, n...
## 1 \r\nUnitedHealthcare Combats Opioid Crisis wit... ... [, unitedhealthcare, combat, opioid, crisis, n...
## 2 The Safety of Chiropractic Adjustments\r\nBy ... ... [, safety, chiropractic, adjustment, lana, bar...
## 3 Advanced Chiropractic Relief: 8 Key Benefits o... ... [advanced, chiropractic, relief, 8, key, benef...
## 4 Heading to the spa can be a pampering treat, b... ... [heading, spa, pampering, treat, also, huge, b...
##
## [5 rows x 10 columns]data.to_csv('dataCleanLemm.csv')
#DATA = pd.read_csv('dataCleanLemm.csv', encoding='unicode_escape')DATA <- read.csv('dataCleanLemm.csv', sep=',', header=TRUE, na.strings=c('',' ','NA'), row.names=1)
colnames(DATA)## [1] "Document" "Source" "Topic"
## [4] "InternetSearch" "Contraindications" "RisksSideEffects"
## [7] "body_length" "punct." "Cleaned_text"
## [10] "Lemmatized"head(DATA,2)## Document
## 0 Chiropractic adjustments and treatments serve the needs of millions of people around the world.\n\n \n\nAdjustments offer effective, non-invasive and cost-effective solutions to neck and back pain, as well as a myriad of other medical issues.\n\nHave you ever stopped to wonder how many of us suffer from neck and back stiffness or pain?\n\nApart from the obvious discomfort, simple daily tasks such as driving a car, crossing a busy street and picking things up from the floor can become all too challenging for individuals experiencing such pain.\n\nAs anyone who has experienced pain would know, having restricted movement can be debilitating and unfortunately, our busy world doesn?t allow for us to stop.\n\n \nSome of the benefits of long-term chiropractic care include:\n\n Chiropractors can identify mechanical issues that cause spine-related pain and offer a series of adjustments that provide near immediate relief. Following appointments, patients often report feeling their symptoms noticeably better.\n When a chiropractor performs an adjustment, they can help restore movement in joints that have ?locked up?. This becomes possible as treatment allows muscles surrounding joints to relax, thereby reducing joint stiffness.\n Many factors affect health, including exercise patterns, nutrition, sleep, heredity and the environment in which we live. Rather than just treat symptoms of the disease, chiropractic care focuses on a holistic approach to naturally maintain health and resist disease.\n Chiropractic adjustments help restore normal function and movement to the entire body. Many patients report an improvement in their ability to move with efficiency and strength.\n Many patients find delight in the results chiropractic adjustments have on old and chronic injuries. Whether an injury is, in fact, new or old, chiropractic care can help reduce pain, restore mobility and provide quick pain relief to all joints in the body. Such care can help maintain better overall health and thus faster recovery time.\n\nHave you ever noticed that when you are in pain and unable to perform regular or favorite activities, it can put a strain on emotional and mental well-being?\n\nFor example, the increased stress from not being able to properly perform a paid job. This, in turn, can have a negative impact on physical health with increases in heart rate and blood pressure. The domino effect often continues with sleep becoming disturbed, with resulting lethargy and tiredness during the day. Does anyone really feel up to exercising in this state?\n\nChiropractic care is a natural method of healing the body?s communication system and never relies on the use of pharmaceutical drugs or invasive surgery.\n\nDoctors of Chiropractic work collaboratively with other healthcare professionals. Should your condition require the attention of another healthcare profession, that recommendation or referral will be made.\n\n
## 1 \nUnitedHealthcare Combats Opioid Crisis with Non-Opioid Benefits\nPhysical therapy and chiropractic care can prevent or reduce expensive, invasive spinal procedures, such as imaging or surgery, to reduce opioid use and cut costs.\nUnitedHealthcare, opioid, physical therapy, healthcare spending\n\nSource: Thinkstock\nShare on Twitter\n\nBy Kelsey Waddill\n\nOctober 29, 2019 - UnitedHealthcare (UHC) is combatting the opioid epidemic and high healthcare costs with new physical therapy and chiropractic care benefits to prevent, delay, or in some cases substitute for invasive spinal procedures.\n\n?With millions of Americans experiencing low back pain currently or at some point during their lifetimes, we believe this benefit design will help make a meaningful difference by improving health outcomes while reducing costs,? said Anne Docimo, MD, UnitedHealthcare chief medical officer.\n\nLower back pain is in part responsible for sustaining the opioid epidemic and also increases healthcare costs.\nDig Deeper\n\n How Major Payers Provide Substance Abuse Care for Opioid Misuse\n Opioid Overdoses Fall by 2% from 2017 to 2018, CDC Reports\n Physical Therapy, Chiropractic Back Care Cut Opioid Use, Costs\n\nAlthough opioid overdoses fell by two percent from 2017 to 2018 and a legal battles aim to hold pharmaceutical companies accountable, there is no end in sight for the opioid epidemic. Industry professionals are still grappling with the balance between cutting opioid prescriptions will working to reduce patient pain.\n\nCommon conditions such as low back pain bolster the epidemic?s presence, with clinicians still prescribing the opioids against best practice recommendations. According to a recent OptumLabs study, 9 percent of patients with newly diagnosed low back pain are prescribed opioids and lower back pain currently contributes 52 percent to the overall opioid prescription rate.\n\nIn addition to boosting opioids distribution, alternative, invasive lower back pain treatments can significantly impact healthcare spending.\n\nIt is not new information that physical therapy and chiropractic care are effective, lower cost alternatives to spinal imaging or surgery. However, payers are still in the process of adopting the method.\n\nTo counteract the high-cost, high-risk potential of using opioids to treat back pain, UHC created a benefit that does not rely on medication or technology but rather on physical therapy and chiropractic care.\n\nThe benefit allows eligible employers to offer physical therapist and chiropractor visits with no out-of-pocket costs. Members who already receive physical therapist and chiropractic care benefits under UHC?s employer-sponsored health plans and who have maxed out their visits will not receive additional visits under this benefit.\n\nHowever, for those who still have visits to use and who choose physical therapy or chiropractic care over other forms of treatment, the copay or deductible for those visits will be waived and they will receive three visits at no cost.\n\nUHC has high expectations for the fiscal and physical impacts of this benefit.\n\nAccording to UHC?s analysis, the health payer expects that by 2021, opioid use will decrease by 19 percent. Spinal imaging test frequency and spinal surgeries will be reduced by 22 percent and 21 percent, respectively. In addition to these specific goals, UHC hopes to see a decrease in the overall cost of spinal care.\n\nThe same OptumLabs study demonstrated that UHC?s expectations are not without precedent.\n\nThe study looked at the correlation between out-of-pocket costs and patient utilization of noninvasive treatments. Researchers discovered that members whose copay was over $30 were a little under 30 percent less likely to choose physical therapy as opposed to more invasive treatments.\n\nAn American Journal of Managed Care study in June 2019 found that patients with high deductibles, typically over $1,000, were less likely to visit physical therapy.\n\nEligible employers may be brand new or renewing their membership. They must be fully insured and over 51 or more employees strong. The benefit is currently available in Connecticut, Florida, Georgia, New York, and North Carolina.\n\nHowever, UHC plans to expand the benefit from 2020 into 2021. By the end of this expansion period, the benefit will also be available to self-funded employers and organizations with an employee population between 2 and 50. The benefit will span ten states, primarily in the southeast.\n\n?This new benefit design may help encourage people with low back pain to get the right care at the right time and in the right setting, helping expand access to evidence-based and more affordable treatments,? said Docimo.
## Source
## 0 https://coremedicalohio.com/benefits-of-long-term-chiropractic-care/?utm_source=ReviveOldPost&utm_medium=social&utm_campaign=ReviveOldPost
## 1 https://healthpayerintelligence.com/news/unitedhealthcare-combats-opioid-crisis-with-non-opioid-benefits
## Topic InternetSearch Contraindications RisksSideEffects
## 0 chiropractic benefits <NA> <NA> <NA>
## 1 chiropractic benefits <NA> <NA> <NA>
## body_length punct.
## 0 2497 2.3
## 1 4055 2.4
## Cleaned_text
## 0 ['chiropract', 'adjust', 'treatment', 'serv', 'need', 'million', 'peopl', 'around', 'world', 'adjust', 'offer', 'effect', 'noninvas', 'costeffect', 'solut', 'neck', 'back', 'pain', 'well', 'myriad', 'medic', 'issu', 'ever', 'stop', 'wonder', 'mani', 'us', 'suffer', 'neck', 'back', 'stiff', 'pain', 'apart', 'obviou', 'discomfort', 'simpl', 'daili', 'task', 'drive', 'car', 'cross', 'busi', 'street', 'pick', 'thing', 'floor', 'becom', 'challeng', 'individu', 'experienc', 'pain', 'anyon', 'experienc', 'pain', 'would', 'know', 'restrict', 'movement', 'debilit', 'unfortun', 'busi', 'world', 'doesnt', 'allow', 'us', 'stop', 'benefit', 'longterm', 'chiropract', 'care', 'includ', 'chiropractor', 'identifi', 'mechan', 'issu', 'caus', 'spinerel', 'pain', 'offer', 'seri', 'adjust', 'provid', 'near', 'immedi', 'relief', 'follow', 'appoint', 'patient', 'often', 'report', 'feel', 'symptom', 'notic', 'better', 'chiropractor', 'perform', 'adjust', 'help', 'restor', 'movement', 'joint', 'lock', 'becom', 'possibl', 'treatment', 'allow', 'muscl', 'surround', 'joint', 'relax', 'therebi', 'reduc', 'joint', 'stiff', 'mani', 'factor', 'affect', 'health', 'includ', 'exercis', 'pattern', 'nutrit', 'sleep', 'hered', 'environ', 'live', 'rather', 'treat', 'symptom', 'diseas', 'chiropract', 'care', 'focus', 'holist', 'approach', 'natur', 'maintain', 'health', 'resist', 'diseas', 'chiropract', 'adjust', 'help', 'restor', 'normal', 'function', 'movement', 'entir', 'bodi', 'mani', 'patient', 'report', 'improv', 'abil', 'move', 'effici', 'strength', 'mani', 'patient', 'find', 'delight', 'result', 'chiropract', 'adjust', 'old', 'chronic', 'injuri', 'whether', 'injuri', 'fact', 'new', 'old', 'chiropract', 'care', 'help', 'reduc', 'pain', 'restor', 'mobil', 'provid', 'quick', 'pain', 'relief', 'joint', 'bodi', 'care', 'help', 'maintain', 'better', 'overal', 'health', 'thu', 'faster', 'recoveri', 'time', 'ever', 'notic', 'pain', 'unabl', 'perform', 'regular', 'favorit', 'activ', 'put', 'strain', 'emot', 'mental', 'wellb', 'exampl', 'increas', 'stress', 'abl', 'properli', 'perform', 'paid', 'job', 'turn', 'neg', 'impact', 'physic', 'health', 'increas', 'heart', 'rate', 'blood', 'pressur', 'domino', 'effect', 'often', 'continu', 'sleep', 'becom', 'disturb', 'result', 'lethargi', 'tired', 'day', 'anyon', 'realli', 'feel', 'exercis', 'state', 'chiropract', 'care', 'natur', 'method', 'heal', 'bodi', 'commun', 'system', 'never', 'reli', 'use', 'pharmaceut', 'drug', 'invas', 'surgeri', 'doctor', 'chiropract', 'work', 'collabor', 'healthcar', 'profession', 'condit', 'requir', 'attent', 'anoth', 'healthcar', 'profess', 'recommend', 'referr', 'made', '']
## 1 ['', 'unitedhealthcar', 'combat', 'opioid', 'crisi', 'nonopioid', 'benefit', 'physic', 'therapi', 'chiropract', 'care', 'prevent', 'reduc', 'expens', 'invas', 'spinal', 'procedur', 'imag', 'surgeri', 'reduc', 'opioid', 'use', 'cut', 'cost', 'unitedhealthcar', 'opioid', 'physic', 'therapi', 'healthcar', 'spend', 'sourc', 'thinkstock', 'share', 'twitter', 'kelsey', 'waddil', 'octob', '29', '2019', 'unitedhealthcar', 'uhc', 'combat', 'opioid', 'epidem', 'high', 'healthcar', 'cost', 'new', 'physic', 'therapi', 'chiropract', 'care', 'benefit', 'prevent', 'delay', 'case', 'substitut', 'invas', 'spinal', 'procedur', 'million', 'american', 'experienc', 'low', 'back', 'pain', 'current', 'point', 'lifetim', 'believ', 'benefit', 'design', 'help', 'make', 'meaning', 'differ', 'improv', 'health', 'outcom', 'reduc', 'cost', 'said', 'ann', 'docimo', 'md', 'unitedhealthcar', 'chief', 'medic', 'offic', 'lower', 'back', 'pain', 'part', 'respons', 'sustain', 'opioid', 'epidem', 'also', 'increas', 'healthcar', 'cost', 'dig', 'deeper', 'major', 'payer', 'provid', 'substanc', 'abus', 'care', 'opioid', 'misus', 'opioid', 'overdos', 'fall', '2', '2017', '2018', 'cdc', 'report', 'physic', 'therapi', 'chiropract', 'back', 'care', 'cut', 'opioid', 'use', 'cost', 'although', 'opioid', 'overdos', 'fell', 'two', 'percent', '2017', '2018', 'legal', 'battl', 'aim', 'hold', 'pharmaceut', 'compani', 'account', 'end', 'sight', 'opioid', 'epidem', 'industri', 'profession', 'still', 'grappl', 'balanc', 'cut', 'opioid', 'prescript', 'work', 'reduc', 'patient', 'pain', 'common', 'condit', 'low', 'back', 'pain', 'bolster', 'epidem', 'presenc', 'clinician', 'still', 'prescrib', 'opioid', 'best', 'practic', 'recommend', 'accord', 'recent', 'optumlab', 'studi', '9', 'percent', 'patient', 'newli', 'diagnos', 'low', 'back', 'pain', 'prescrib', 'opioid', 'lower', 'back', 'pain', 'current', 'contribut', '52', 'percent', 'overal', 'opioid', 'prescript', 'rate', 'addit', 'boost', 'opioid', 'distribut', 'altern', 'invas', 'lower', 'back', 'pain', 'treatment', 'significantli', 'impact', 'healthcar', 'spend', 'new', 'inform', 'physic', 'therapi', 'chiropract', 'care', 'effect', 'lower', 'cost', 'altern', 'spinal', 'imag', 'surgeri', 'howev', 'payer', 'still', 'process', 'adopt', 'method', 'counteract', 'highcost', 'highrisk', 'potenti', 'use', 'opioid', 'treat', 'back', 'pain', 'uhc', 'creat', 'benefit', 'reli', 'medic', 'technolog', 'rather', 'physic', 'therapi', 'chiropract', 'care', 'benefit', 'allow', 'elig', 'employ', 'offer', 'physic', 'therapist', 'chiropractor', 'visit', 'outofpocket', 'cost', 'member', 'alreadi', 'receiv', 'physic', 'therapist', 'chiropract', 'care', 'benefit', 'uhc', 'employersponsor', 'health', 'plan', 'max', 'visit', 'receiv', 'addit', 'visit', 'benefit', 'howev', 'still', 'visit', 'use', 'choos', 'physic', 'therapi', 'chiropract', 'care', 'form', 'treatment', 'copay', 'deduct', 'visit', 'waiv', 'receiv', 'three', 'visit', 'cost', 'uhc', 'high', 'expect', 'fiscal', 'physic', 'impact', 'benefit', 'accord', 'uhc', 'analysi', 'health', 'payer', 'expect', '2021', 'opioid', 'use', 'decreas', '19', 'percent', 'spinal', 'imag', 'test', 'frequenc', 'spinal', 'surgeri', 'reduc', '22', 'percent', '21', 'percent', 'respect', 'addit', 'specif', 'goal', 'uhc', 'hope', 'see', 'decreas', 'overal', 'cost', 'spinal', 'care', 'optumlab', 'studi', 'demonstr', 'uhc', 'expect', 'without', 'preced', 'studi', 'look', 'correl', 'outofpocket', 'cost', 'patient', 'util', 'noninvas', 'treatment', 'research', 'discov', 'member', 'whose', 'copay', '30', 'littl', '30', 'percent', 'less', 'like', 'choos', 'physic', 'therapi', 'oppos', 'invas', 'treatment', 'american', 'journal', 'manag', 'care', 'studi', 'june', '2019', 'found', 'patient', 'high', 'deduct', 'typic', '1000', 'less', 'like', 'visit', 'physic', 'therapi', 'elig', 'employ', 'may', 'brand', 'new', 'renew', 'membership', 'must', 'fulli', 'insur', '51', 'employe', 'strong', 'benefit', 'current', 'avail', 'connecticut', 'florida', 'georgia', 'new', 'york', 'north', 'carolina', 'howev', 'uhc', 'plan', 'expand', 'benefit', '2020', '2021', 'end', 'expans', 'period', 'benefit', 'also', 'avail', 'selffund', 'employ', 'organ', 'employe', 'popul', '2', '50', 'benefit', 'span', 'ten', 'state', 'primarili', 'southeast', 'new', 'benefit', 'design', 'may', 'help', 'encourag', 'peopl', 'low', 'back', 'pain', 'get', 'right', 'care', 'right', 'time', 'right', 'set', 'help', 'expand', 'access', 'evidencebas', 'afford', 'treatment', 'said', 'docimo']
## Lemmatized
## 0 ['chiropractic', 'adjustment', 'treatment', 'serve', 'need', 'million', 'people', 'around', 'world', 'adjustment', 'offer', 'effective', 'noninvasive', 'costeffective', 'solution', 'neck', 'back', 'pain', 'well', 'myriad', 'medical', 'issue', 'ever', 'stopped', 'wonder', 'many', 'u', 'suffer', 'neck', 'back', 'stiffness', 'pain', 'apart', 'obvious', 'discomfort', 'simple', 'daily', 'task', 'driving', 'car', 'crossing', 'busy', 'street', 'picking', 'thing', 'floor', 'become', 'challenging', 'individual', 'experiencing', 'pain', 'anyone', 'experienced', 'pain', 'would', 'know', 'restricted', 'movement', 'debilitating', 'unfortunately', 'busy', 'world', 'doesnt', 'allow', 'u', 'stop', 'benefit', 'longterm', 'chiropractic', 'care', 'include', 'chiropractor', 'identify', 'mechanical', 'issue', 'cause', 'spinerelated', 'pain', 'offer', 'series', 'adjustment', 'provide', 'near', 'immediate', 'relief', 'following', 'appointment', 'patient', 'often', 'report', 'feeling', 'symptom', 'noticeably', 'better', 'chiropractor', 'performs', 'adjustment', 'help', 'restore', 'movement', 'joint', 'locked', 'becomes', 'possible', 'treatment', 'allows', 'muscle', 'surrounding', 'joint', 'relax', 'thereby', 'reducing', 'joint', 'stiffness', 'many', 'factor', 'affect', 'health', 'including', 'exercise', 'pattern', 'nutrition', 'sleep', 'heredity', 'environment', 'live', 'rather', 'treat', 'symptom', 'disease', 'chiropractic', 'care', 'focus', 'holistic', 'approach', 'naturally', 'maintain', 'health', 'resist', 'disease', 'chiropractic', 'adjustment', 'help', 'restore', 'normal', 'function', 'movement', 'entire', 'body', 'many', 'patient', 'report', 'improvement', 'ability', 'move', 'efficiency', 'strength', 'many', 'patient', 'find', 'delight', 'result', 'chiropractic', 'adjustment', 'old', 'chronic', 'injury', 'whether', 'injury', 'fact', 'new', 'old', 'chiropractic', 'care', 'help', 'reduce', 'pain', 'restore', 'mobility', 'provide', 'quick', 'pain', 'relief', 'joint', 'body', 'care', 'help', 'maintain', 'better', 'overall', 'health', 'thus', 'faster', 'recovery', 'time', 'ever', 'noticed', 'pain', 'unable', 'perform', 'regular', 'favorite', 'activity', 'put', 'strain', 'emotional', 'mental', 'wellbeing', 'example', 'increased', 'stress', 'able', 'properly', 'perform', 'paid', 'job', 'turn', 'negative', 'impact', 'physical', 'health', 'increase', 'heart', 'rate', 'blood', 'pressure', 'domino', 'effect', 'often', 'continues', 'sleep', 'becoming', 'disturbed', 'resulting', 'lethargy', 'tiredness', 'day', 'anyone', 'really', 'feel', 'exercising', 'state', 'chiropractic', 'care', 'natural', 'method', 'healing', 'body', 'communication', 'system', 'never', 'relies', 'use', 'pharmaceutical', 'drug', 'invasive', 'surgery', 'doctor', 'chiropractic', 'work', 'collaboratively', 'healthcare', 'professional', 'condition', 'require', 'attention', 'another', 'healthcare', 'profession', 'recommendation', 'referral', 'made', '']
## 1 ['', 'unitedhealthcare', 'combat', 'opioid', 'crisis', 'nonopioid', 'benefit', 'physical', 'therapy', 'chiropractic', 'care', 'prevent', 'reduce', 'expensive', 'invasive', 'spinal', 'procedure', 'imaging', 'surgery', 'reduce', 'opioid', 'use', 'cut', 'cost', 'unitedhealthcare', 'opioid', 'physical', 'therapy', 'healthcare', 'spending', 'source', 'thinkstock', 'share', 'twitter', 'kelsey', 'waddill', 'october', '29', '2019', 'unitedhealthcare', 'uhc', 'combatting', 'opioid', 'epidemic', 'high', 'healthcare', 'cost', 'new', 'physical', 'therapy', 'chiropractic', 'care', 'benefit', 'prevent', 'delay', 'case', 'substitute', 'invasive', 'spinal', 'procedure', 'million', 'american', 'experiencing', 'low', 'back', 'pain', 'currently', 'point', 'lifetime', 'believe', 'benefit', 'design', 'help', 'make', 'meaningful', 'difference', 'improving', 'health', 'outcome', 'reducing', 'cost', 'said', 'anne', 'docimo', 'md', 'unitedhealthcare', 'chief', 'medical', 'officer', 'lower', 'back', 'pain', 'part', 'responsible', 'sustaining', 'opioid', 'epidemic', 'also', 'increase', 'healthcare', 'cost', 'dig', 'deeper', 'major', 'payer', 'provide', 'substance', 'abuse', 'care', 'opioid', 'misuse', 'opioid', 'overdoses', 'fall', '2', '2017', '2018', 'cdc', 'report', 'physical', 'therapy', 'chiropractic', 'back', 'care', 'cut', 'opioid', 'use', 'cost', 'although', 'opioid', 'overdoses', 'fell', 'two', 'percent', '2017', '2018', 'legal', 'battle', 'aim', 'hold', 'pharmaceutical', 'company', 'accountable', 'end', 'sight', 'opioid', 'epidemic', 'industry', 'professional', 'still', 'grappling', 'balance', 'cutting', 'opioid', 'prescription', 'working', 'reduce', 'patient', 'pain', 'common', 'condition', 'low', 'back', 'pain', 'bolster', 'epidemic', 'presence', 'clinician', 'still', 'prescribing', 'opioids', 'best', 'practice', 'recommendation', 'according', 'recent', 'optumlabs', 'study', '9', 'percent', 'patient', 'newly', 'diagnosed', 'low', 'back', 'pain', 'prescribed', 'opioids', 'lower', 'back', 'pain', 'currently', 'contributes', '52', 'percent', 'overall', 'opioid', 'prescription', 'rate', 'addition', 'boosting', 'opioids', 'distribution', 'alternative', 'invasive', 'lower', 'back', 'pain', 'treatment', 'significantly', 'impact', 'healthcare', 'spending', 'new', 'information', 'physical', 'therapy', 'chiropractic', 'care', 'effective', 'lower', 'cost', 'alternative', 'spinal', 'imaging', 'surgery', 'however', 'payer', 'still', 'process', 'adopting', 'method', 'counteract', 'highcost', 'highrisk', 'potential', 'using', 'opioids', 'treat', 'back', 'pain', 'uhc', 'created', 'benefit', 'rely', 'medication', 'technology', 'rather', 'physical', 'therapy', 'chiropractic', 'care', 'benefit', 'allows', 'eligible', 'employer', 'offer', 'physical', 'therapist', 'chiropractor', 'visit', 'outofpocket', 'cost', 'member', 'already', 'receive', 'physical', 'therapist', 'chiropractic', 'care', 'benefit', 'uhcs', 'employersponsored', 'health', 'plan', 'maxed', 'visit', 'receive', 'additional', 'visit', 'benefit', 'however', 'still', 'visit', 'use', 'choose', 'physical', 'therapy', 'chiropractic', 'care', 'form', 'treatment', 'copay', 'deductible', 'visit', 'waived', 'receive', 'three', 'visit', 'cost', 'uhc', 'high', 'expectation', 'fiscal', 'physical', 'impact', 'benefit', 'according', 'uhcs', 'analysis', 'health', 'payer', 'expects', '2021', 'opioid', 'use', 'decrease', '19', 'percent', 'spinal', 'imaging', 'test', 'frequency', 'spinal', 'surgery', 'reduced', '22', 'percent', '21', 'percent', 'respectively', 'addition', 'specific', 'goal', 'uhc', 'hope', 'see', 'decrease', 'overall', 'cost', 'spinal', 'care', 'optumlabs', 'study', 'demonstrated', 'uhcs', 'expectation', 'without', 'precedent', 'study', 'looked', 'correlation', 'outofpocket', 'cost', 'patient', 'utilization', 'noninvasive', 'treatment', 'researcher', 'discovered', 'member', 'whose', 'copay', '30', 'little', '30', 'percent', 'le', 'likely', 'choose', 'physical', 'therapy', 'opposed', 'invasive', 'treatment', 'american', 'journal', 'managed', 'care', 'study', 'june', '2019', 'found', 'patient', 'high', 'deductible', 'typically', '1000', 'le', 'likely', 'visit', 'physical', 'therapy', 'eligible', 'employer', 'may', 'brand', 'new', 'renewing', 'membership', 'must', 'fully', 'insured', '51', 'employee', 'strong', 'benefit', 'currently', 'available', 'connecticut', 'florida', 'georgia', 'new', 'york', 'north', 'carolina', 'however', 'uhc', 'plan', 'expand', 'benefit', '2020', '2021', 'end', 'expansion', 'period', 'benefit', 'also', 'available', 'selffunded', 'employer', 'organization', 'employee', 'population', '2', '50', 'benefit', 'span', 'ten', 'state', 'primarily', 'southeast', 'new', 'benefit', 'design', 'may', 'help', 'encourage', 'people', 'low', 'back', 'pain', 'get', 'right', 'care', 'right', 'time', 'right', 'setting', 'helping', 'expand', 'access', 'evidencebased', 'affordable', 'treatment', 'said', 'docimo']from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(data[['Document','body_length','punct%','Cleaned_text','Lemmatized']],data['Topic'],test_size=0.15)from sklearn.feature_extraction.text import CountVectorizercount_vect=CountVectorizer(analyzer=lemmatize)
count_vect_fit=count_vect.fit(X_train['Document'])
count_train=count_vect_fit.transform(X_train['Document'])
count_test=count_vect_fit.transform(X_test['Document'])len(count_vect_fit.get_feature_names())## 4697count_vect_fit.get_feature_names()[200:350]## ['acting', 'action', 'activates', 'activating', 'activation', 'active', 'actively', 'activity', 'actual', 'actually', 'actuallyyou', 'acupoints', 'acupressure', 'acupuncture', 'acupuncturist', 'acute', 'ad', 'adapted', 'add', 'added', 'addiction', 'adding', 'addition', 'additional', 'additionally', 'address', 'addressed', 'addressing', 'addthis', 'adeline', 'adequate', 'adhesion', 'adjunct', 'adjustable', 'adjustment', 'administered', 'administering', 'administration', 'adminmcb', 'admission', 'admitted', 'adolescent', 'adopted', 'adopting', 'adore', 'adult', 'adults1', 'advanced', 'advantage', 'advantageous', 'adverse', 'adversity', 'advertised', 'advertisement', 'advice', 'advise', 'advisory', 'advocate', 'aerobic', 'affair', 'affect', 'affected', 'affecting', 'affiliate', 'affirm', 'afford', 'affordable', 'afterhours', 'afterward', 'afterwards', 'agakian', 'age', 'agency', 'agephysical', 'agerelated', 'aggression', 'aggressive', 'aggressively', 'agility', 'aging', 'ago', 'agree', 'ahead', 'aid', 'aiding', 'ailment', 'aim', 'aimed', 'air', 'airport', 'aka', 'al', 'alchemist', 'alcohol', 'alert', 'aligning', 'alignment', 'alike', 'allergic', 'allergy', 'alleviate', 'alleviates', 'alleviating', 'alliance', 'allison', 'allow', 'allowing', 'allows', 'almost', 'alone', 'along', 'alongside', 'already', 'alright', 'also', 'alter', 'altered', 'alternate', 'alternating', 'alternatingly', 'alternative', 'alters', 'although', 'altogether', 'always', 'alzheimers', 'amateur', 'amazing', 'amazon', 'amazoncom', 'ambulance', 'ameliorating', 'america', 'american', 'among', 'amount', 'amplitude', 'amta', 'amy', 'analysis', 'analyzed', 'anatomy', 'ancient', 'andor', 'anecdotal', 'anemia', 'anesthetic', 'angeles', 'anger', 'angle']count_train_vect=pd.concat([X_train[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(count_train.toarray())],axis=1)
count_test_vect=pd.concat([X_test[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(count_test.toarray())],axis=1)count_train_vect.head()## Document body_length ... 4695 4696
## 0 Cupping Therapy\r\nPrivacy & Trust Info\r\nCor... 7684 ... 1 0
## 1 \r\nFive Warning Signs of Mental Illness\r\n\r... 6991 ... 0 0
## 2 This futuristic ?gun? could be more effective ... 5308 ... 0 0
## 3 lucy should experience the mindnumbing pleasur... 49 ... 0 0
## 4 Learn All About Cupping Therapy And The Many ... 2204 ... 0 0
##
## [5 rows x 4702 columns]from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import precision_recall_fscore_support as score
import timerf=RandomForestClassifier(n_estimators=150, max_depth=None, n_jobs=-1)
start=time.time()
rf_model=rf.fit(count_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=rf_model.predict(count_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 3 chiropractic benefits chiropractic benefits
## 78 Not Professional Not Professional
## 2 chiropractic benefits chiropractic benefits
## 35 cupping benefits cupping benefits
## 62 cold stone benefits cold stone benefits
## 51 massage gun benefits massage gun benefits
## 68 Not Professional Not Professional
## 80 Not Professional Not Professional
## 4 massage benefits massage benefits
## 8 Not Professional massage benefits
## 69 Not Professional Not Professional
## 71 Not Professional Not Professional
## 20 physical therapy benefits physical therapy benefits
## 5 massage benefits massage benefitsfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.9285714285714286print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[5 0 0 0 0 0 0]
## [0 2 0 0 0 0 0]
## [0 0 1 0 0 0 0]
## [0 0 0 1 0 0 0]
## [1 0 0 0 2 0 0]
## [0 0 0 0 0 1 0]
## [0 0 0 0 0 0 1]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 0.83 1.00 0.91 5
## chiropractic benefits 1.00 1.00 1.00 2
## cold stone benefits 1.00 1.00 1.00 1
## cupping benefits 1.00 1.00 1.00 1
## massage benefits 1.00 0.67 0.80 3
## massage gun benefits 1.00 1.00 1.00 1
## physical therapy benefits 1.00 1.00 1.00 1
##
## accuracy 0.93 14
## macro avg 0.98 0.95 0.96 14
## weighted avg 0.94 0.93 0.92 14gb=GradientBoostingClassifier(n_estimators=150,max_depth=11)
start=time.time()
gb_model=gb.fit(count_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=gb_model.predict(count_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 3 chiropractic benefits chiropractic benefits
## 78 Not Professional Not Professional
## 2 chiropractic benefits chiropractic benefits
## 35 cupping benefits cupping benefits
## 62 cold stone benefits cold stone benefits
## 51 massage gun benefits massage gun benefits
## 68 Not Professional Not Professional
## 80 Not Professional Not Professional
## 4 massage benefits massage benefits
## 8 Not Professional massage benefits
## 69 Not Professional Not Professional
## 71 Not Professional Not Professional
## 20 physical therapy benefits physical therapy benefits
## 5 massage benefits massage benefitsfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.9285714285714286print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[5 0 0 0 0 0 0]
## [0 2 0 0 0 0 0]
## [0 0 1 0 0 0 0]
## [0 0 0 1 0 0 0]
## [1 0 0 0 2 0 0]
## [0 0 0 0 0 1 0]
## [0 0 0 0 0 0 1]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 0.83 1.00 0.91 5
## chiropractic benefits 1.00 1.00 1.00 2
## cold stone benefits 1.00 1.00 1.00 1
## cupping benefits 1.00 1.00 1.00 1
## massage benefits 1.00 0.67 0.80 3
## massage gun benefits 1.00 1.00 1.00 1
## physical therapy benefits 1.00 1.00 1.00 1
##
## accuracy 0.93 14
## macro avg 0.98 0.95 0.96 14
## weighted avg 0.94 0.93 0.92 14TF-IDF RFC and GBC
stopwords = nltk.corpus.stopwords.words('english')
ps=nltk.PorterStemmer()
wn=nltk.WordNetLemmatizer()
data=pd.read_csv("benefitsContraindications3.csv", encoding='unicode_escape')
data.columns=['Document','Source','Topic','InternetSearch','Contraindications','RisksSideEffects']def count_punct(text):
count=sum([1 for char in text if char in string.punctuation])
return round(count/(len(text)-text.count(" ")),3)*100data['body_length']=data['Document'].apply(lambda x: len(x)-x.count(" "))
data['punct%']= data['Document'].apply(lambda x: count_punct(x))
def clean_text(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=[ps.stem(word) for word in tokens if word not in stopwords]#these are lists for the count vectorizer
return text
def lemmatize(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=[wn.lemmatize(word) for word in tokens if word not in stopwords]#these are lists for the count vectorizer
return text
data['Cleaned_text']=data['Document'].apply(lambda x: clean_text(x))
data['Lemmatized']=data['Document'].apply(lambda x: lemmatize(x))
data.head()## Document ... Lemmatized
## 0 Chiropractic adjustments and treatments serve ... ... [chiropractic, adjustment, treatment, serve, n...
## 1 \r\nUnitedHealthcare Combats Opioid Crisis wit... ... [, unitedhealthcare, combat, opioid, crisis, n...
## 2 The Safety of Chiropractic Adjustments\r\nBy ... ... [, safety, chiropractic, adjustment, lana, bar...
## 3 Advanced Chiropractic Relief: 8 Key Benefits o... ... [advanced, chiropractic, relief, 8, key, benef...
## 4 Heading to the spa can be a pampering treat, b... ... [heading, spa, pampering, treat, also, huge, b...
##
## [5 rows x 10 columns]
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(data[['Document','body_length','punct%','Cleaned_text','Lemmatized']],data['Topic'],test_size=0.15)
from sklearn.feature_extraction.text import TfidfVectorizer
tfidf_vect=TfidfVectorizer(analyzer=lemmatize)
tfidf_vect_fit=tfidf_vect.fit(X_train['Document'])
tfidf_train=tfidf_vect_fit.transform(X_train['Document'])
tfidf_test=tfidf_vect_fit.transform(X_test['Document'])len(tfidf_vect_fit.get_feature_names())## 4729tfidf_vect_fit.get_feature_names()[200:350]## ['active', 'actively', 'activity', 'actual', 'actually', 'actuallyyou', 'acupoints', 'acupressure', 'acupuncture', 'acupuncturist', 'acupuncturists', 'acute', 'ad', 'add', 'added', 'addiction', 'adding', 'addition', 'additional', 'additionally', 'address', 'addressed', 'addressing', 'addthis', 'adeline', 'adequate', 'adhesion', 'adjunct', 'adjust', 'adjustable', 'adjusted', 'adjusting', 'adjustment', 'administered', 'administering', 'administration', 'adminmcb', 'admission', 'admitted', 'adolescent', 'adopted', 'adopting', 'adore', 'adult', 'adultery', 'adults1', 'advanced', 'advantage', 'advantageous', 'adverse', 'adversity', 'advertised', 'advertisement', 'advice', 'advise', 'advisory', 'advocate', 'aerobic', 'affair', 'affect', 'affected', 'affecting', 'affiliate', 'affirm', 'afford', 'affordable', 'afraid', 'afterhours', 'afterward', 'age', 'agency', 'agerelated', 'aggression', 'aggressive', 'aggressively', 'agility', 'aging', 'ago', 'agree', 'ahead', 'aid', 'aiding', 'ailment', 'aim', 'aimed', 'air', 'airport', 'ajman', 'al', 'alchemist', 'alcohol', 'alcoholsoaked', 'alert', 'align', 'aligning', 'alignment', 'alike', 'allergic', 'allergy', 'alleviate', 'alleviates', 'alleviating', 'alleviation', 'alliance', 'allison', 'allow', 'allowing', 'allows', 'almost', 'alone', 'along', 'alongside', 'already', 'alright', 'also', 'alter', 'altered', 'alternate', 'alternating', 'alternative', 'alters', 'although', 'altogether', 'always', 'alzheimers', 'amateur', 'amazing', 'amazon', 'amazoncom', 'ambulance', 'ameliorating', 'america', 'american', 'among', 'amount', 'amplitude', 'amta', 'amy', 'analgesic', 'analysis', 'analyzed', 'anatomy', 'ancient', 'andor', 'anecdotal', 'anemia', 'anesthetic', 'angeles', 'anger', 'angle']tfidf_train_vect=pd.concat([X_train[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(tfidf_train.toarray())],axis=1)
tfidf_test_vect=pd.concat([X_test[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(tfidf_test.toarray())],axis=1)tfidf_train_vect.head()## Document body_length ... 4727 4728
## 0 7 Benefits of Massage Therapy\r\n\r\nMassage t... 5948 ... 0.0 0.0
## 1 What Is Cupping Therapy?\r\n\r\n Cupping ty... 4344 ... 0.0 0.0
## 2 \r\n21 Benefits of Chiropractic Adjustments\r\... 1546 ... 0.0 0.0
## 3 ll You Need To Know About Massage Gun\r\n31 Au... 4598 ... 0.0 0.0
## 4 Goodnight everyone, say it back or Ill massag... 66 ... 0.0 0.0
##
## [5 rows x 4734 columns]from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import precision_recall_fscore_support as score
import time
np.random.seed(45678)rf=RandomForestClassifier(n_estimators=150, max_depth=None, n_jobs=-1)
start=time.time()
rf_model=rf.fit(tfidf_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=rf_model.predict(tfidf_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 79 Not Professional Not Professional
## 20 physical therapy benefits physical therapy benefits
## 38 cupping benefits cupping benefits
## 60 cold stone benefits cold stone benefits
## 63 cold stone benefits cold stone benefits
## 73 Not Professional Not Professional
## 17 physical therapy benefits physical therapy benefits
## 59 cold stone benefits cold stone benefits
## 76 Not Professional Not Professional
## 15 massage benefits massage benefits
## 19 Not Professional physical therapy benefits
## 18 Not Professional physical therapy benefits
## 57 cold stone benefits cold stone benefits
## 67 Not Professional Not Professionalfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.8571428571428571print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[4 0 0 0 0]
## [0 4 0 0 0]
## [0 0 1 0 0]
## [0 0 0 1 0]
## [2 0 0 0 2]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 0.67 1.00 0.80 4
## cold stone benefits 1.00 1.00 1.00 4
## cupping benefits 1.00 1.00 1.00 1
## massage benefits 1.00 1.00 1.00 1
## physical therapy benefits 1.00 0.50 0.67 4
##
## accuracy 0.86 14
## macro avg 0.93 0.90 0.89 14
## weighted avg 0.90 0.86 0.85 14gb=GradientBoostingClassifier(n_estimators=150,max_depth=11)
start=time.time()
gb_model=gb.fit(tfidf_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=gb_model.predict(tfidf_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 79 massage benefits Not Professional
## 20 physical therapy benefits physical therapy benefits
## 38 cupping benefits cupping benefits
## 60 cold stone benefits cold stone benefits
## 63 cold stone benefits cold stone benefits
## 73 massage benefits Not Professional
## 17 physical therapy benefits physical therapy benefits
## 59 cold stone benefits cold stone benefits
## 76 Not Professional Not Professional
## 15 massage benefits massage benefits
## 19 massage benefits physical therapy benefits
## 18 physical therapy benefits physical therapy benefits
## 57 cold stone benefits cold stone benefits
## 67 Not Professional Not Professionalfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.7857142857142857print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[2 0 0 2 0]
## [0 4 0 0 0]
## [0 0 1 0 0]
## [0 0 0 1 0]
## [0 0 0 1 3]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 1.00 0.50 0.67 4
## cold stone benefits 1.00 1.00 1.00 4
## cupping benefits 1.00 1.00 1.00 1
## massage benefits 0.25 1.00 0.40 1
## physical therapy benefits 1.00 0.75 0.86 4
##
## accuracy 0.79 14
## macro avg 0.85 0.85 0.78 14
## weighted avg 0.95 0.79 0.82 14N-Grams Vectorization for RFC and GBC
stopwords = nltk.corpus.stopwords.words('english')
ps=nltk.PorterStemmer()
wn=nltk.WordNetLemmatizer()
data=pd.read_csv("benefitsContraindications3.csv", encoding='unicode_escape')
data.columns=['Document','Source','Topic','InternetSearch','Contraindications','RisksSideEffects']def count_punct(text):
count=sum([1 for char in text if char in string.punctuation])
return round(count/(len(text)-text.count(" ")),3)*100data['body_length']=data['Document'].apply(lambda x: len(x)-x.count(" "))
data['punct%']= data['Document'].apply(lambda x: count_punct(x))def clean_text(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+',text)
text=" ".join([ps.stem(word) for word in tokens if word not in stopwords])#unlisted with N-grams vectorization
return text
def lemmatize(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=" ".join([wn.lemmatize(word) for word in tokens if word not in stopwords])#unlisted with N-grams vectorization
#text=[wn.lemmatize(word) for word in tokens if word not in stopwords]#when using count Vectorization its a list
#or else single letters returned.
return text data['Cleaned_text']=data['Document'].apply(lambda x: clean_text(x))
data['Lemmatized']=data['Document'].apply(lambda x: lemmatize(x))
data.head()## Document ... Lemmatized
## 0 Chiropractic adjustments and treatments serve ... ... chiropractic adjustment treatment serve need m...
## 1 \r\nUnitedHealthcare Combats Opioid Crisis wit... ... unitedhealthcare combat opioid crisis nonopio...
## 2 The Safety of Chiropractic Adjustments\r\nBy ... ... safety chiropractic adjustment lana barhum me...
## 3 Advanced Chiropractic Relief: 8 Key Benefits o... ... advanced chiropractic relief 8 key benefit chi...
## 4 Heading to the spa can be a pampering treat, b... ... heading spa pampering treat also huge boost he...
##
## [5 rows x 10 columns]from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(data[['Document','body_length','punct%','Cleaned_text','Lemmatized']],data['Topic'],test_size=0.15)from sklearn.feature_extraction.text import CountVectorizern_gram_vect=CountVectorizer(ngram_range=(1,4))type(X_train['Cleaned_text'])## <class 'pandas.core.series.Series'>X_train['Cleaned_text'].head()## 33 chiropract care back pain articl chiropract c...
## 32 chiropract care back pain articl chiropract c...
## 18 physic therapi chronic pain expect articl phy...
## 84 know need go er coronaviru sometim stay home r...
## 69 avail good massag plu servic sex top nice romant
## Name: Cleaned_text, dtype: objectX_train['Lemmatized'].head()## 33 chiropractic care back pain article chiroprac...
## 32 chiropractic care back pain article chiroprac...
## 18 physical therapy chronic pain expect article ...
## 84 know need go er coronavirus sometimes staying ...
## 69 available good massage plus service sex top ni...
## Name: Lemmatized, dtype: objectn_gram_vect_fit=n_gram_vect.fit(X_train['Lemmatized'])
n_gram_train=n_gram_vect_fit.transform(X_train['Lemmatized'])
n_gram_test=n_gram_vect_fit.transform(X_test['Lemmatized'])len(n_gram_vect_fit.get_feature_names())## 77224print(n_gram_vect_fit.get_feature_names()[200:500])## ['183 patient', '183 patient neck', '183 patient neck pain', '18772228387', '18772228387 already', '18772228387 already benefit', '18772228387 already benefit va', '18999', '18999 amazoncom', '18999 amazoncom damkees', '18999 amazoncom damkees professional', '18999 damkee', '18999 damkee currently', '18999 damkee currently sale', '18h', '18h lmassagebodytobody', '18h lmassagebodytobody massage', '18h lmassagebodytobody massage massage', '19', '19 percent', '19 percent spinal', '19 percent spinal imaging', '192', '192 people', '192 people backrelated', '192 people backrelated pain', '1950s', '1950s across', '1950s across hospital', '1950s across hospital china', '1950s cupping', '1950s cupping also', '1950s cupping also practiced', '1996', '1996 study', '1996 study rand', '1996 study rand corporation', '19999', '19999 amazoncom', '19999 amazoncom 4000', '19999 amazoncom 4000 amazon', '20', '20 2015', '20 2015 therapy', '20 2015 therapy comment', '20 different', '20 different adjustable', '20 different adjustable speed', '20 minute', '20 minute depending', '20 minute depending nature', '20 minute long', '20 minute long say', '20 year', '20 year without', '20 year without adverse', '2000', '2000 3000', '2000 3000 hour', '2000 3000 hour supervised', '2001', '2001 overview', '2001 overview scoring', '2001 overview scoring phq9', '2001 study', '2001 study published', '2001 study published canadian', '2003', '2003 evaluated', '2003 evaluated 183', '2003 evaluated 183 patient', '2003 scored', '2003 scored might', '2003 scored might proceed', '2005', '2005 study', '2005 study published', '2005 study published international', '2007', '2007 according', '2007 according national', '2007 according national health', '2007 article', '2007 article chicago', '2007 article chicago tribune', '2007 likely', '2007 likely seek', '2007 likely seek coaching', '2007 stress', '2007 stress also', '2007 stress also lead', '2008', '2008 center', '2008 center disease', '2008 center disease control', '2008 grown', '2008 grown become', '2008 grown become favorite', '2008 researcher', '2008 researcher evaluated', '2008 researcher evaluated chiropractic', '2009', '2009 disease', '2009 disease link', '2009 disease link inflammation', '2009 found', '2009 found reflective', '2009 found reflective journaling', '2009 reduction', '2009 reduction chronic', '2009 reduction chronic pain', '2010', '2010 study', '2010 study published', '2010 study published journal', '2011', '2011 cnnmoneycom', '2011 cnnmoneycom gave', '2011 cnnmoneycom gave physical', '2011 number', '2011 number adult', '2011 number adult seek', '2011 review', '2011 review 26', '2011 review 26 clinical', '2011 study', '2011 study published', '2011 study published annals', '2012', '2012 report', '2012 report published', '2012 report published plo', '2012 review', '2012 review studiestrusted', '2012 review studiestrusted source', '2012 well', '2012 well benefit', '2012 well benefit society', '2013', '2013 2011', '2013 2011 cnnmoneycom', '2013 2011 cnnmoneycom gave', '2013 illustrating', '2013 illustrating benefit', '2013 illustrating benefit journaling', '2014', '2014 2024a', '2014 2024a much', '2014 2024a much quicker', '2014 study', '2014 study suggested', '2014 study suggested vibration', '2015', '2015 journal', '2015 journal traditional', '2015 journal traditional complementary', '2015 report', '2015 report published', '2015 report published journal', '2015 review', '2015 review evidence', '2015 review evidence found', '2015 therapy', '2015 therapy comment', '2015 therapy comment physical', '2015patricia', '2015patricia mayrhofer', '2015patricia mayrhofer spatechnique', '2015patricia mayrhofer spatechnique articles2', '2016', '2016 930', '2016 930 robert', '2016 930 robert shmerling', '2016 cold', '2016 cold stone', '2016 cold stone massage', '2016 michael', '2016 michael phelps', '2016 michael phelps permanently', '2016 mr', '2016 mr castaneda', '2016 mr castaneda written', '2016 study', '2016 study published', '2016 study published journal', '2016 summer', '2016 summer olympics', '2016 summer olympics share', '2016 used', '2016 used cupping', '2016 used cupping easily', '2017', '2017 2018', '2017 2018 cdc', '2017 2018 cdc report', '2017 2018 legal', '2017 2018 legal battle', '2017 chiropractor', '2017 chiropractor tout', '2017 chiropractor tout treatment', '2017 nba', '2017 nba final', '2017 nba final irving', '2017 study', '2017 study found', '2017 study found structure', '2018', '2018 84000', '2018 84000 topic', '2018 84000 topic physical', '2018 according', '2018 according american', '2018 according american massage', '2018 cdc', '2018 cdc report', '2018 cdc report physical', '2018 finding', '2018 finding help', '2018 finding help comment', '2018 galluppalmer', '2018 galluppalmer college', '2018 galluppalmer college chiropractic', '2018 getty', '2018 getty image', '2018 getty image cupping', '2018 individual', '2018 individual salary', '2018 individual salary vary', '2018 largest', '2018 largest percentage', '2018 largest percentage working', '2018 legal', '2018 legal battle', '2018 legal battle aim', '2018 study', '2018 study led', '2018 study led dr', '2019', '2019 1011', '2019 1011 article', '2019 1011 article based', '2019 1122', '2019 1122 share', '2019 1122 share article', '2019 342', '2019 342 pm', '2019 342 pm edt', '2019 early', '2019 early 2020', '2019 early 2020 many', '2019 found', '2019 found patient', '2019 found patient high', '2019 massage', '2019 massage gun', '2019 massage gun one', '2019 physical', '2019 physical therapist', '2019 physical therapist stretch', '2019 puhhhagetty', '2019 puhhhagetty image', '2019 puhhhagetty image come', '2019 rich', '2019 rich barnesgetty', '2019 rich barnesgetty image', '2019 story', '2019 story previously', '2019 story previously published', '2019 thai', '2019 thai massage', '2019 thai massage full', '2019 unitedhealthcare', '2019 unitedhealthcare uhc', '2019 unitedhealthcare uhc combatting', '2020', '2020 2021', '2020 2021 end', '2020 2021 end expansion', '2020 beyond', '2020 beyond people', '2020 beyond people say', '2020 click', '2020 click read', '2020 click read user', '2020 depressed', '2020 depressed man', '2020 depressed man lying', '2020 karen', '2020 karen green', '2020 karen green last', '2020 many', '2020 many people', '2020 many people started', '2020 may', '2020 may look', '2020 may look like', '2020 orthopedics', '2020 orthopedics orthopedics', '2020 orthopedics orthopedics home', '2020 spring']n_gram_train_df=pd.concat([X_train[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(n_gram_train.toarray())],axis=1)
n_gram_test_df=pd.concat([X_test[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(n_gram_test.toarray())],axis=1)n_gram_train_df.head()## Document body_length ... 77222 77223
## 0 \r\nChiropractic Care for Back Pain\r\nIn this... 507 ... 0 0
## 1 \r\nChiropractic Care for Back Pain\r\nIn this... 2366 ... 0 0
## 2 \r\nPhysical Therapy for Chronic Pain: What to... 1854 ... 0 0
## 3 How to Know If You Need to Go to the E.R. With... 8779 ... 0 0
## 4 Available now.. good massage plus service sex.... 68 ... 0 0
##
## [5 rows x 77229 columns]from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import precision_recall_fscore_support as score
import timerf=RandomForestClassifier(n_estimators=150, max_depth=None, n_jobs=-1)
start=time.time()
rf_model=rf.fit(n_gram_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=rf_model.predict(n_gram_test)
end=time.time()
pred_time=(end-start)
prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 3 massage benefits chiropractic benefits
## 41 cupping benefits cupping benefits
## 62 Not Professional cold stone benefits
## 48 Not Professional massage gun benefits
## 43 Not Professional cupping benefits
## 70 Not Professional Not Professional
## 25 Not Professional mental health services benefits
## 75 Not Professional Not Professional
## 21 physical therapy benefits physical therapy benefits
## 15 Not Professional massage benefits
## 77 Not Professional Not Professional
## 55 massage benefits cold stone benefits
## 11 Not Professional massage benefits
## 36 Not Professional cupping benefitsfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.35714285714285715print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[3 0 0 0 0 0 0 0]
## [0 0 0 0 1 0 0 0]
## [1 0 0 0 1 0 0 0]
## [2 0 0 1 0 0 0 0]
## [2 0 0 0 0 0 0 0]
## [1 0 0 0 0 0 0 0]
## [1 0 0 0 0 0 0 0]
## [0 0 0 0 0 0 0 1]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 0.30 1.00 0.46 3
## chiropractic benefits 0.00 0.00 0.00 1
## cold stone benefits 0.00 0.00 0.00 2
## cupping benefits 1.00 0.33 0.50 3
## massage benefits 0.00 0.00 0.00 2
## massage gun benefits 0.00 0.00 0.00 1
## mental health services benefits 0.00 0.00 0.00 1
## physical therapy benefits 1.00 1.00 1.00 1
##
## accuracy 0.36 14
## macro avg 0.29 0.29 0.25 14
## weighted avg 0.35 0.36 0.28 14
##
##
## C:\Users\m\Anaconda2\envs\python36\lib\site-packages\sklearn\metrics\classification.py:1437: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.
## 'precision', 'predicted', average, warn_for)gb=GradientBoostingClassifier(n_estimators=150,max_depth=11)
start=time.time()
gb_model=gb.fit(n_gram_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=gb_model.predict(n_gram_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 3 chiropractic benefits chiropractic benefits
## 41 cupping benefits cupping benefits
## 62 cold stone benefits cold stone benefits
## 48 massage gun benefits massage gun benefits
## 43 cupping benefits cupping benefits
## 70 Not Professional Not Professional
## 25 mental health services benefits mental health services benefits
## 75 Not Professional Not Professional
## 21 physical therapy benefits physical therapy benefits
## 15 massage benefits massage benefits
## 77 Not Professional Not Professional
## 55 massage benefits cold stone benefits
## 11 massage benefits massage benefits
## 36 cupping benefits cupping benefitsfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.9285714285714286print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[3 0 0 0 0 0 0 0]
## [0 1 0 0 0 0 0 0]
## [0 0 1 0 1 0 0 0]
## [0 0 0 3 0 0 0 0]
## [0 0 0 0 2 0 0 0]
## [0 0 0 0 0 1 0 0]
## [0 0 0 0 0 0 1 0]
## [0 0 0 0 0 0 0 1]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 1.00 1.00 1.00 3
## chiropractic benefits 1.00 1.00 1.00 1
## cold stone benefits 1.00 0.50 0.67 2
## cupping benefits 1.00 1.00 1.00 3
## massage benefits 0.67 1.00 0.80 2
## massage gun benefits 1.00 1.00 1.00 1
## mental health services benefits 1.00 1.00 1.00 1
## physical therapy benefits 1.00 1.00 1.00 1
##
## accuracy 0.93 14
## macro avg 0.96 0.94 0.93 14
## weighted avg 0.95 0.93 0.92 14Third part: Stemmed Tokens & 80/20 Train/Test split & RFC | GBC
Count Vectorizer RFC and GBC
stopwords = nltk.corpus.stopwords.words('english')
ps=nltk.PorterStemmer()
wn=nltk.WordNetLemmatizer()
data=pd.read_csv("benefitsContraindications3.csv", encoding='unicode_escape')
data.columns=['Document','Source','Topic','InternetSearch','Contraindications','RisksSideEffects']data <- read.csv('benefitsContraindications3.csv',sep=',',header=TRUE, na.strings=c('',' ','NA'))
colnames(data)## [1] "Document" "Source" "Topic"
## [4] "InternetSearch" "Contraindications" "risksAdverseEffects"head(data,5)## Document
## 1 Chiropractic adjustments and treatments serve the needs of millions of people around the world.\n\n \n\nAdjustments offer effective, non-invasive and cost-effective solutions to neck and back pain, as well as a myriad of other medical issues.\n\nHave you ever stopped to wonder how many of us suffer from neck and back stiffness or pain?\n\nApart from the obvious discomfort, simple daily tasks such as driving a car, crossing a busy street and picking things up from the floor can become all too challenging for individuals experiencing such pain.\n\nAs anyone who has experienced pain would know, having restricted movement can be debilitating and unfortunately, our busy world doesn?t allow for us to stop.\n\n \nSome of the benefits of long-term chiropractic care include:\n\n Chiropractors can identify mechanical issues that cause spine-related pain and offer a series of adjustments that provide near immediate relief. Following appointments, patients often report feeling their symptoms noticeably better.\n When a chiropractor performs an adjustment, they can help restore movement in joints that have ?locked up?. This becomes possible as treatment allows muscles surrounding joints to relax, thereby reducing joint stiffness.\n Many factors affect health, including exercise patterns, nutrition, sleep, heredity and the environment in which we live. Rather than just treat symptoms of the disease, chiropractic care focuses on a holistic approach to naturally maintain health and resist disease.\n Chiropractic adjustments help restore normal function and movement to the entire body. Many patients report an improvement in their ability to move with efficiency and strength.\n Many patients find delight in the results chiropractic adjustments have on old and chronic injuries. Whether an injury is, in fact, new or old, chiropractic care can help reduce pain, restore mobility and provide quick pain relief to all joints in the body. Such care can help maintain better overall health and thus faster recovery time.\n\nHave you ever noticed that when you are in pain and unable to perform regular or favorite activities, it can put a strain on emotional and mental well-being?\n\nFor example, the increased stress from not being able to properly perform a paid job. This, in turn, can have a negative impact on physical health with increases in heart rate and blood pressure. The domino effect often continues with sleep becoming disturbed, with resulting lethargy and tiredness during the day. Does anyone really feel up to exercising in this state?\n\nChiropractic care is a natural method of healing the body?s communication system and never relies on the use of pharmaceutical drugs or invasive surgery.\n\nDoctors of Chiropractic work collaboratively with other healthcare professionals. Should your condition require the attention of another healthcare profession, that recommendation or referral will be made.\n\n
## 2 \nUnitedHealthcare Combats Opioid Crisis with Non-Opioid Benefits\nPhysical therapy and chiropractic care can prevent or reduce expensive, invasive spinal procedures, such as imaging or surgery, to reduce opioid use and cut costs.\nUnitedHealthcare, opioid, physical therapy, healthcare spending\n\nSource: Thinkstock\nShare on Twitter\n\nBy Kelsey Waddill\n\nOctober 29, 2019 - UnitedHealthcare (UHC) is combatting the opioid epidemic and high healthcare costs with new physical therapy and chiropractic care benefits to prevent, delay, or in some cases substitute for invasive spinal procedures.\n\n?With millions of Americans experiencing low back pain currently or at some point during their lifetimes, we believe this benefit design will help make a meaningful difference by improving health outcomes while reducing costs,? said Anne Docimo, MD, UnitedHealthcare chief medical officer.\n\nLower back pain is in part responsible for sustaining the opioid epidemic and also increases healthcare costs.\nDig Deeper\n\n How Major Payers Provide Substance Abuse Care for Opioid Misuse\n Opioid Overdoses Fall by 2% from 2017 to 2018, CDC Reports\n Physical Therapy, Chiropractic Back Care Cut Opioid Use, Costs\n\nAlthough opioid overdoses fell by two percent from 2017 to 2018 and a legal battles aim to hold pharmaceutical companies accountable, there is no end in sight for the opioid epidemic. Industry professionals are still grappling with the balance between cutting opioid prescriptions will working to reduce patient pain.\n\nCommon conditions such as low back pain bolster the epidemic?s presence, with clinicians still prescribing the opioids against best practice recommendations. According to a recent OptumLabs study, 9 percent of patients with newly diagnosed low back pain are prescribed opioids and lower back pain currently contributes 52 percent to the overall opioid prescription rate.\n\nIn addition to boosting opioids distribution, alternative, invasive lower back pain treatments can significantly impact healthcare spending.\n\nIt is not new information that physical therapy and chiropractic care are effective, lower cost alternatives to spinal imaging or surgery. However, payers are still in the process of adopting the method.\n\nTo counteract the high-cost, high-risk potential of using opioids to treat back pain, UHC created a benefit that does not rely on medication or technology but rather on physical therapy and chiropractic care.\n\nThe benefit allows eligible employers to offer physical therapist and chiropractor visits with no out-of-pocket costs. Members who already receive physical therapist and chiropractic care benefits under UHC?s employer-sponsored health plans and who have maxed out their visits will not receive additional visits under this benefit.\n\nHowever, for those who still have visits to use and who choose physical therapy or chiropractic care over other forms of treatment, the copay or deductible for those visits will be waived and they will receive three visits at no cost.\n\nUHC has high expectations for the fiscal and physical impacts of this benefit.\n\nAccording to UHC?s analysis, the health payer expects that by 2021, opioid use will decrease by 19 percent. Spinal imaging test frequency and spinal surgeries will be reduced by 22 percent and 21 percent, respectively. In addition to these specific goals, UHC hopes to see a decrease in the overall cost of spinal care.\n\nThe same OptumLabs study demonstrated that UHC?s expectations are not without precedent.\n\nThe study looked at the correlation between out-of-pocket costs and patient utilization of noninvasive treatments. Researchers discovered that members whose copay was over $30 were a little under 30 percent less likely to choose physical therapy as opposed to more invasive treatments.\n\nAn American Journal of Managed Care study in June 2019 found that patients with high deductibles, typically over $1,000, were less likely to visit physical therapy.\n\nEligible employers may be brand new or renewing their membership. They must be fully insured and over 51 or more employees strong. The benefit is currently available in Connecticut, Florida, Georgia, New York, and North Carolina.\n\nHowever, UHC plans to expand the benefit from 2020 into 2021. By the end of this expansion period, the benefit will also be available to self-funded employers and organizations with an employee population between 2 and 50. The benefit will span ten states, primarily in the southeast.\n\n?This new benefit design may help encourage people with low back pain to get the right care at the right time and in the right setting, helping expand access to evidence-based and more affordable treatments,? said Docimo.
## 3 The Safety of Chiropractic Adjustments\nBy Lana Barhum\nMedically reviewed by Richard N. Fogoros, MD\nUpdated on January 31, 2020\nOrthopedics\nMore in Orthopedics\n\n Home Office Ergonomics\n Sprains & Strains\n Fractures & Broken Bones\n Physical Therapy\n Orthopedic Surgery\n Osteoporosis\n Pediatric Orthopedics\n Sports Injuries\n\nView All\nIn This Article\n\n Chiropractic Adjustment\n What Research Shows\n Safety\n\nChiropractic adjustment, also called spinal manipulation, is a procedure done by a chiropractor using the hands or small instruments to apply controlled force to a spinal joint. The goal is to improve spinal motion and physical function of the entire body. Chiropractic adjustment is safe when performed by someone who is properly trained and licensed to practice chiropractic care. Complications are rare, but they are possible. Learn more about both the benefits and risks.\nChiropractic adjustment\nVerywell / Brianna Gilmartin \nChiropractic Adjustment\n\nOne of the most important reasons people seek chiropractic care is because it is a completely drug-free therapy. Someone dealing with joint pain, back pain, or headaches might consider visiting a chiropractor.\n\nThe goal of chiropractic adjustment is to place the body into a proper position so the body can heal itself. Treatments are believed to reduce stress on the immune system, reducing the potential for disease. Chiropractic care aims to address the entire body, including a person?s ability to move, perform, and even think.\nWhat Research Shows\n\nMany people wonder how helpful chiropractic care is in treating years of trauma and poor posture. There have been numerous studies showing the therapeutic benefits of chiropractic care.\nSciatica\n\nSciatica is a type of pain affecting the sciatic nerve, the large nerve extending from the low back down the back of the legs. Other natural therapies don?t always offer relief and most people want to avoid steroid injections and surgery, so they turn to chiropractic care.\n\nA double-blind trial reported in the Spine Journal compared active and simulated chiropractic manipulations in people with sciatic nerve pain. Active manipulations involved the patient laying down and receiving treatment from a chiropractor. Stimulated manipulations involved electrical muscle stimulation with electrodes placed on the skin to send electrical pulses to different parts of the body.\n\nThe researchers determined active manipulation offered more benefits than stimulated. The people who received active manipulations experienced fewer days of moderate or severe pain and other sciatica symptoms. They also reported no adverse effects.\nNeck Pain\n\nOne study reported in the Annals of Internal Medicine looked at different therapies for treating neck pain. They divided 272 study participants into three groups: one that received spinal manipulation from a chiropractic doctor, a second group given over-the-counter (OTC) pain relievers, narcotics, and muscle relaxers, and a third group who did at-home exercises. \n\nAfter 12 weeks, patients reported a 75% pain reduction, with the chiropractic treatment group achieving the most improvement. About 57% of the chiropractic group achieved pain reduction, while 48% received pain reduction from exercising, and 33% from medication.\n\nAfter one year, 53% of the drug-free groups continued to report pain relief compared to only 38% of those taking pain medications. \nHeadaches\n\nCervicogenic headaches and migraines are commonly treated by chiropractors. Cervicogenic headaches are often called secondary headaches because pain is usually referred from another source, usually the neck. Migraine headaches cause severe, throbbing pain and are generally experienced on one side of the head. There are few non-medicinal options for managing both types of chronic headaches.\n\nResearch reported in the Journal of Manipulative and Physiological Therapeutics suggests chiropractic care, specifically spinal manipulation, can improve migraines and cervicogenic headaches. \nFrozen Shoulder\n\nFrozen shoulder affects the shoulder joint and involves pain and stiffness that develops gradually and gets worse. Frozen shoulder can be quite painful, and treatment involves preserving as much range of motion in the shoulder as possible and managing pain.\n\nA clinical trial reported in the Journal of Chiropractic Medicine described how patients suffering from frozen shoulder responded to chiropractic treatment. Of the 50 patients, 16 completely recovered, 25 showed a 75 to 90% improvement, and eight showed a 50 to 75% improvement. Only one person showed zero to 50% improvement. The researchers concluded most people can get improvement by treating frozen shoulder with chiropractic treatment.\nPreventing Need for Surgery\n\nChiropractic care may reduce the need for back surgery. Guidelines reported in the Journal of the American Medical Association suggest that it's reasonable for people suffering from back pain to try spinal manipulation before deciding on surgical intervention.\nLow Back Pain\n\nStudies have shown chiropractic care, including spinal manipulation, can provide relief from mild to moderate low back pain. In fact, spinal manipulation may work as well as other standard treatments, including pain-relief medications.\n\nA 2011 review of 26 clinical trials looked at the effectiveness of different treatments for chronic low back pain. What they found was that spinal manipulation is just as effective as other treatments for reducing back pain and improving function.\nSafety\n\n\nWhen chiropractors are correctly trained and licensed, chiropractic care is safe. Mild side effects are to be expected and include temporary soreness, stiffness, and tenderness in the treated area. However, you still want to do your research. Ask for a referral from your doctor. Look at the chiropractor?s website, including patient reviews. Meet with the chiropractor to discuss his or her treatment practices and ask about possible adverse effects related to treatment.\n\nIf you decide a chiropractor isn?t for you, consider seeing an osteopathic doctor. Osteopaths are fully licensed doctors who can practice all areas of medicine. They have received special training on the musculoskeletal system, which includes manual readjustments, myofascial release and other physical manipulation of bones and muscle tissues.
## 4 Advanced Chiropractic Relief: 8 Key Benefits of Chiropractor Care\n\nAre you one of the 50 million Americans who suffer from chronic pain? If so you?re probably intimately familiar with the feeling of pure desperation that can arise from an inability to find relief.\n\nIn addition to physical issues, chronic pain can cause anxiety, depression, and more. However, there could be a light at the end of the tunnel. Many people are finding advanced chiropractic relief that is completely changing their lives.\n\nYour body is a world in itself. At this very moment, more than a million chemical reactions are taking place in your body. It manufactures energy, it regulates your heartbeat, your breathing and it regenerates and heals itself. Everything takes place without your conscious knowledge, without you controlling it voluntarily. The master system that controls it all is your nervous system.\n\nThe nervous system is made out of your brain, spinal cord and all your nerves.\n\nThe energy that flows through your nervous system in your body is like electricity. In order to have that electric flow normally and freely, we need to have a well functioning spine. Whenever you have disruption of that flow, disease happens. That would be the case when your spine is misaligned or is not moving properly.\n\nDid you know that 90% of stimulation and nutrition to the brain is generated by the movement of the spine?\n\nThe more mechanically distorted a person is, the less energy is available for thinking, metabolism and healing.\n\nThis is why it is so important to have a healthy spine, a proper posture, to exercise, to eat properly ? all of it truly matters for your quality of life.\nChiropractors localize the areas of your spine that do not move properly ? referred to as vertebral subluxations ? and adjust them with a specific high speed, but yet gentle, thrust to improve spinal motion.\n\nWant to learn about some of the ways chiropractic care can help you? Keep reading for insight into some of the key benefits of seeing a chiropractor.\n\nThe benefits of chiropractic care are numerous:\n\n1. Lower Blood Pressure\n\nStudies show that chiropractic treatment can lower your blood pressure. Sometimes, this works just as well as a prescription blood pressure medication! This benefit can also last for as long as six months after treatment.\n\nHigh blood pressure can cause an array of serious side effects like nausea, fatigue, dizziness, and anxiety. Sufferers who haven?t found relief should consider consulting with a chiropractor. A chiropractic adjustment may be the solution.\n\nSome studies have shown that chiropractic adjustments can also help patients who are suffering from low blood pressure.\n\n2. Reduced Inflammation\n\nIn many cases, joint issues, pain, and tension are caused by inflammation in the body. Chiropractic adjustments can reduce inflammation.\n\nThis leads to relief of muscle tension, chronic back pain, and joint pain. These adjustments can sometimes also slow the progression of inflammation-related diseases, like arthritis.\n\n3. Better Sleep\n\nPatients who receive chiropractic adjustments report a significant improvement in their sleep patterns. If you regularly suffer from insomnia, visiting a chiropractor regularly may help. Also, when you experience pain relief, this will help you get a restful night?s sleep.\n\n4. Digestive Relief\n\nChiropractors often give nutritional advice as part of their services. However, this isn?t the only way that they provide patients with digestive relief.\n\nAdjusting the thoraco-lumbar spine restores the neurological function of your digestive system. Regular adjustments can help with chronic digestive issues.\n\n5. Stress Release\n\nEveryday life can cause muscle cramping, inflammation, and more. When you?re sore from working at a computer, heavy lifting, or just dealing with emotional stress, a chiropractic adjustment can help. This leads to greater comfort and advanced pain relief.\n\n6. Improvement of Neurological Conditions\n\nA chiropractic adjustment can also increase blood flow to the brain and increase the flow of cerebral spinal fluid. This means that patients suffering from neurological conditions like epilepsy and multiple sclerosis can significantly benefit from regular adjustments.\n\nThis is a relatively new area of study, but the potential is huge. Those suffering from these conditions will want to do some research. It?s important to find the best chiropractor in their area with experience dealing with these specific types of cases.\n\n7. Chiropractic care can improve communication from your brain to your muscles\n\nResearch seems to show that chiropractic care can improve your brain-body communication, helping your brain to be more aware of what is going on in the body so it can control your body better.\n\nBetter health, more energy and vitality are some of the positive effects of getting your spine adjusted. It sets your vertebrae back into motion freeing up the energy that travels through your nerves.\n\nChiropractic care is a partnership. The results patients want is a combination of what the chiropractor does and what the patient does.\n\nThere are many good things that can be changed and improved for a better lifestyle: exercise, good nutrition, good mental attitude and spinal adjustments.\n\nYour whole body will work better by having your nervous system free of interference. That is the essence of chiropractic care and is designed for you and your family.\n\n8. Pain Relief\n\nPerhaps the most well-known benefit of going to a chiropractor is pain relief. Adjustments can help with a huge array of painful conditions including the following.\n\nNeck and Lower Back Pain\n\nAdjustments are the most effective non-invasive pain relief method for this type of pain. They may help patients avoid having to take prescription pain management drugs.\n\nSciatica\n\nTreatments help relieve pressure on the nerve. This results in less severe pain that lasts for a fewer number of days.\n\nHeadaches\n\nChiropractic adjustments help headaches and migraines. They do this by treating back misalignment, muscle tension, and stress. Cervical spine manipulation was associated with significant improvement in headache outcomes in trials involving patients with neck pain and/or neck dysfunction and headache.\n\nChronic headaches can result from the abnormal positioning of the head and can be worsened from neck pressure and movement. Chiropractic removes the interference whether it may be from the distant muscle tightness in the back causing strain on your spine or an abnormal lordotic cervical curve and moving vertebrae.\nChiropractic care can reduce the duration of headaches, lower their intensity when they do occur and limit the frequency of their occurrence all together.\n\nMenstrual cramps\n\nChiropractic treatment removes tension from the pelvis and sacrum. It also regulates the neurological function communicating with the reproductive organs. Adjustments can also relieve the bloating, cramping, and pain associated with menstrual cramps\n\nAnyone who has tried traditional medical treatments and has been unable to find pain relief should experiment with chiropractic care. More often than not, you?ll be pleasantly surprised!\n\nBonus: Advanced Chiropractic Relief\n\nIn addition to the benefits listed above, adjustments can bring advanced chiropractic relief for a wide variety of other conditions as well as overall life improvement. A few examples include:\n\nScoliosis ? adjustments have shown to help with the pain, reduced range of motion, abnormal posture, and even difficulty breathing caused by this abnormal curvature of the spine\n\nVertigo ? an adjustment can help realign and balance the spine, thereby reducing the dizziness, nausea, and disorientation caused by vertigo\n\nSinus and allergy relief ? adjusting the upper cervical spine can help drain the sinuses and provide immediate and lasting relief from both long-term and seasonal allergies\n\nExpectant mothers ? women can experience relief from pain and morning sickness and are better able to maintain proper posture during and after pregnancy\n\nChildren?s issues ? treatments have been shown to help children with acid reflux, cholic, and ear infections\nAthletic performance ? the reduction in pain and inflammation is particularly beneficial for professional and amateur athletes\n\nStimulates the immune system ? chiropractic care helps to boost the immune system, speeding up the healing process following illnesses or injuries. One of the most important studies showing the positive effect chiropractic care can have on the immune system and general health was performed by Ronald Pero, Ph.D., chief of cancer prevention research at New York?s Preventive Medicine Institute and professor of medicine at New York University. Dr. Pero measured the immune systems of people under chiropractic care as compared to those in the general population and those with cancer and other serious diseases.\n\nIn his initial three-year study of 107 individuals who had been under chiropractic care for five years or more, the chiropractic patients were found to have a 200% greater immune competence than people who had not received chiropractic care, and 400% greater immune competence than people with cancer and other serious diseases. The immune system superiority of those under chiropractic care did not diminish with age.\n\nDr. Pero stated: ?When applied in a clinical framework, I have never seen a group other than this chiropractic group to experience a 200% increase over the normal patients. This is why it is so dramatically important. We have never seen such a positive improvement in a group.?\n\nAs you can see, there are almost limitless benefits to seeking chiropractic treatment. If you haven?t tried it yet, what are you waiting for?\n\nThere?s no need to accept pain and discomfort as a normal part of life. You have nothing to lose and everything to gain, so it only makes sense to find out more about this possibly life-changing approach to improving your health and wellness.
## 5 Heading to the spa can be a pampering treat, but it can also be a huge boost to your health and wellness! Massage therapy can relieve all sorts of ailments ? from physical pain, to stress and anxiety. People who choose to supplement their healthcare regimen with regular massages will not only enjoy a relaxing hour or two at the spa, but they will see the benefits carry through the days and weeks after the appointment!\n\n1\n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\nThese are the 10 most common benefits reported from massage therapy:\n\n1. Reduce Stress\n\nA relaxing day at the spa is a great way to unwind and de-stress. However, clients are sure to notice themselves feeling relaxed and at ease for days and even weeks after their appointments!\n\n \n\n2. Improve Circulation\n\nLoosening muscles and tendons allows increased blood flow throughout the body. Improving your circulation can have a number of positive effects on the rest of your body, including reduced fatigue and pain management!\n\n \n\n3. Reduce Pain\n\nMassage therapy is great for working out problem areas like lower back pain and chronic stiffness. A professional therapist will be able to accurately target the source of your pain and help achieve the perfect massage regimen.\n\n \n\n3\n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n4. Eliminate Toxins\n\nStimulating the soft tissues of your body will help to release toxins through your blood and lymphatic systems.\n\n \n\n5. Improve Flexibility\n\nMassage therapy will loosen and relax your muscles, helping your body to achieve its full range of movement potential.\n\n \n\n6. Improve Sleep\n\nA massage will encourage relaxation and boost your mood. Going to bed with relaxed and loosened muscles promotes more restful sleep, and you?ll feel less tired in the morning!\n\n \n\n7. Enhance Immunity\n\nStimulation of the lymph nodes re-charges the body?s natural defense system.\n\n \n\n8. Reduce Fatigue\n\nMassage therapy is known to boost mood and promote better quality sleep, thus making you feel more rested and less worn-out at the end of the day.\n\n \n\n2\n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n9. Alleviate Depression and Anxiety\n\nMassage therapy can help to release endorphins in your body, helping you to feel happy, energized, and at ease.\n\n \n\n10. Reduce post-surgery and post-injury swelling\n\nA professional massage is a great way to safely deal with a sports injury or post-surgery rehabilitation.\n\nDo you think that massage therapy could help you find relief in any of these areas? What improvements would you like to see in your health? Contact us today with your questions about massage therapy and see how we can help you get on the path to improved health and wellness!
## Source
## 1 https://coremedicalohio.com/benefits-of-long-term-chiropractic-care/?utm_source=ReviveOldPost&utm_medium=social&utm_campaign=ReviveOldPost
## 2 https://healthpayerintelligence.com/news/unitedhealthcare-combats-opioid-crisis-with-non-opioid-benefits
## 3 https://www.verywellhealth.com/is-chiropractic-adjustment-safe-4588279
## 4 https://hafkeychiropractic.com/advanced-chiropractic-relief-8-key-benefits-of-chiropractor-care/
## 5 https://www.urbannirvana.com/10-benefits-massage-therapy/
## Topic InternetSearch Contraindications
## 1 chiropractic benefits <NA> <NA>
## 2 chiropractic benefits <NA> <NA>
## 3 chiropractic benefits <NA> <NA>
## 4 chiropractic benefits <NA> <NA>
## 5 massage benefits google <NA>
## risksAdverseEffects
## 1 <NA>
## 2 <NA>
## 3 Risks and side effects associated with chiropractic adjustments may include:\n\n temporary headaches\n fatigue after treatment\n discomfort in parts of the body that were treated\n\nRare but serious risks associated with chiropractic adjustment include:\n\n stroke\n cauda equina syndrome, a condition involving pinched nerves in the lower part of the spinal canal\n worsening of herniated disks (although research isn't conclusive)\n\nIn addition to effectiveness, research has focused on the safety of chiropractic treatments, mainly spinal manipulation. \n\nOne 2017 review of 250 articles looked at serious adverse events and benign events associated with chiropractic care. Based on the evidence the researchers reviewed, serious adverse events accounted for one out of every two million spinal manipulations to 13 per 10,00 patients. Serious adverse events included spinal or neurological problems and cervical arterial strokes (dissection of any of the arteries in the neck).\n\nBenign events were more common and included more pain and higher levels of neck problems, but most were short-term problems.\n\nThe researchers confirmed serious adverse events were rare and often related to other preexisting conditions, while benign events are more common. However, the reasons for any types of adverse events are unknown.\n\nA second 2017 review looked 118 articles and found frequently described adverse events include stroke, headache and vertebral artery dissection (cervical arterial stroke). Forty-six percent of the reviews determined that spinal manipulation was safe, while 13% expressed concern of harm. The remaining studies were unclear or neutral. While the researchers did not offer an overall conclusion, they determined spinal manipulation can significantly be helpful, and some risk does exist.\nA Word From Verywell
## 4 <NA>
## 5 <NA>def count_punct(text):
count=sum([1 for char in text if char in string.punctuation])
return round(count/(len(text)-text.count(" ")),3)*100data['body_length']=data['Document'].apply(lambda x: len(x)-x.count(" "))
data['punct%']= data['Document'].apply(lambda x: count_punct(x))
def clean_text(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=[ps.stem(word) for word in tokens if word not in stopwords]#these are lists for the count vectorizer
return text
def lemmatize(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=[wn.lemmatize(word) for word in tokens if word not in stopwords]#these are lists for the count vectorizer
return text
data['Cleaned_text']=data['Document'].apply(lambda x: clean_text(x))
data['Lemmatized']=data['Document'].apply(lambda x: lemmatize(x))
data.head()## Document ... Lemmatized
## 0 Chiropractic adjustments and treatments serve ... ... [chiropractic, adjustment, treatment, serve, n...
## 1 \r\nUnitedHealthcare Combats Opioid Crisis wit... ... [, unitedhealthcare, combat, opioid, crisis, n...
## 2 The Safety of Chiropractic Adjustments\r\nBy ... ... [, safety, chiropractic, adjustment, lana, bar...
## 3 Advanced Chiropractic Relief: 8 Key Benefits o... ... [advanced, chiropractic, relief, 8, key, benef...
## 4 Heading to the spa can be a pampering treat, b... ... [heading, spa, pampering, treat, also, huge, b...
##
## [5 rows x 10 columns]data.to_csv('dataCleanLemm.csv')
#DATA = pd.read_csv('dataCleanLemm.csv', encoding='unicode_escape')DATA <- read.csv('dataCleanLemm.csv', sep=',', header=TRUE, na.strings=c('',' ','NA'), row.names=1)
colnames(DATA)## [1] "Document" "Source" "Topic"
## [4] "InternetSearch" "Contraindications" "RisksSideEffects"
## [7] "body_length" "punct." "Cleaned_text"
## [10] "Lemmatized"head(DATA,2)## Document
## 0 Chiropractic adjustments and treatments serve the needs of millions of people around the world.\n\n \n\nAdjustments offer effective, non-invasive and cost-effective solutions to neck and back pain, as well as a myriad of other medical issues.\n\nHave you ever stopped to wonder how many of us suffer from neck and back stiffness or pain?\n\nApart from the obvious discomfort, simple daily tasks such as driving a car, crossing a busy street and picking things up from the floor can become all too challenging for individuals experiencing such pain.\n\nAs anyone who has experienced pain would know, having restricted movement can be debilitating and unfortunately, our busy world doesn?t allow for us to stop.\n\n \nSome of the benefits of long-term chiropractic care include:\n\n Chiropractors can identify mechanical issues that cause spine-related pain and offer a series of adjustments that provide near immediate relief. Following appointments, patients often report feeling their symptoms noticeably better.\n When a chiropractor performs an adjustment, they can help restore movement in joints that have ?locked up?. This becomes possible as treatment allows muscles surrounding joints to relax, thereby reducing joint stiffness.\n Many factors affect health, including exercise patterns, nutrition, sleep, heredity and the environment in which we live. Rather than just treat symptoms of the disease, chiropractic care focuses on a holistic approach to naturally maintain health and resist disease.\n Chiropractic adjustments help restore normal function and movement to the entire body. Many patients report an improvement in their ability to move with efficiency and strength.\n Many patients find delight in the results chiropractic adjustments have on old and chronic injuries. Whether an injury is, in fact, new or old, chiropractic care can help reduce pain, restore mobility and provide quick pain relief to all joints in the body. Such care can help maintain better overall health and thus faster recovery time.\n\nHave you ever noticed that when you are in pain and unable to perform regular or favorite activities, it can put a strain on emotional and mental well-being?\n\nFor example, the increased stress from not being able to properly perform a paid job. This, in turn, can have a negative impact on physical health with increases in heart rate and blood pressure. The domino effect often continues with sleep becoming disturbed, with resulting lethargy and tiredness during the day. Does anyone really feel up to exercising in this state?\n\nChiropractic care is a natural method of healing the body?s communication system and never relies on the use of pharmaceutical drugs or invasive surgery.\n\nDoctors of Chiropractic work collaboratively with other healthcare professionals. Should your condition require the attention of another healthcare profession, that recommendation or referral will be made.\n\n
## 1 \nUnitedHealthcare Combats Opioid Crisis with Non-Opioid Benefits\nPhysical therapy and chiropractic care can prevent or reduce expensive, invasive spinal procedures, such as imaging or surgery, to reduce opioid use and cut costs.\nUnitedHealthcare, opioid, physical therapy, healthcare spending\n\nSource: Thinkstock\nShare on Twitter\n\nBy Kelsey Waddill\n\nOctober 29, 2019 - UnitedHealthcare (UHC) is combatting the opioid epidemic and high healthcare costs with new physical therapy and chiropractic care benefits to prevent, delay, or in some cases substitute for invasive spinal procedures.\n\n?With millions of Americans experiencing low back pain currently or at some point during their lifetimes, we believe this benefit design will help make a meaningful difference by improving health outcomes while reducing costs,? said Anne Docimo, MD, UnitedHealthcare chief medical officer.\n\nLower back pain is in part responsible for sustaining the opioid epidemic and also increases healthcare costs.\nDig Deeper\n\n How Major Payers Provide Substance Abuse Care for Opioid Misuse\n Opioid Overdoses Fall by 2% from 2017 to 2018, CDC Reports\n Physical Therapy, Chiropractic Back Care Cut Opioid Use, Costs\n\nAlthough opioid overdoses fell by two percent from 2017 to 2018 and a legal battles aim to hold pharmaceutical companies accountable, there is no end in sight for the opioid epidemic. Industry professionals are still grappling with the balance between cutting opioid prescriptions will working to reduce patient pain.\n\nCommon conditions such as low back pain bolster the epidemic?s presence, with clinicians still prescribing the opioids against best practice recommendations. According to a recent OptumLabs study, 9 percent of patients with newly diagnosed low back pain are prescribed opioids and lower back pain currently contributes 52 percent to the overall opioid prescription rate.\n\nIn addition to boosting opioids distribution, alternative, invasive lower back pain treatments can significantly impact healthcare spending.\n\nIt is not new information that physical therapy and chiropractic care are effective, lower cost alternatives to spinal imaging or surgery. However, payers are still in the process of adopting the method.\n\nTo counteract the high-cost, high-risk potential of using opioids to treat back pain, UHC created a benefit that does not rely on medication or technology but rather on physical therapy and chiropractic care.\n\nThe benefit allows eligible employers to offer physical therapist and chiropractor visits with no out-of-pocket costs. Members who already receive physical therapist and chiropractic care benefits under UHC?s employer-sponsored health plans and who have maxed out their visits will not receive additional visits under this benefit.\n\nHowever, for those who still have visits to use and who choose physical therapy or chiropractic care over other forms of treatment, the copay or deductible for those visits will be waived and they will receive three visits at no cost.\n\nUHC has high expectations for the fiscal and physical impacts of this benefit.\n\nAccording to UHC?s analysis, the health payer expects that by 2021, opioid use will decrease by 19 percent. Spinal imaging test frequency and spinal surgeries will be reduced by 22 percent and 21 percent, respectively. In addition to these specific goals, UHC hopes to see a decrease in the overall cost of spinal care.\n\nThe same OptumLabs study demonstrated that UHC?s expectations are not without precedent.\n\nThe study looked at the correlation between out-of-pocket costs and patient utilization of noninvasive treatments. Researchers discovered that members whose copay was over $30 were a little under 30 percent less likely to choose physical therapy as opposed to more invasive treatments.\n\nAn American Journal of Managed Care study in June 2019 found that patients with high deductibles, typically over $1,000, were less likely to visit physical therapy.\n\nEligible employers may be brand new or renewing their membership. They must be fully insured and over 51 or more employees strong. The benefit is currently available in Connecticut, Florida, Georgia, New York, and North Carolina.\n\nHowever, UHC plans to expand the benefit from 2020 into 2021. By the end of this expansion period, the benefit will also be available to self-funded employers and organizations with an employee population between 2 and 50. The benefit will span ten states, primarily in the southeast.\n\n?This new benefit design may help encourage people with low back pain to get the right care at the right time and in the right setting, helping expand access to evidence-based and more affordable treatments,? said Docimo.
## Source
## 0 https://coremedicalohio.com/benefits-of-long-term-chiropractic-care/?utm_source=ReviveOldPost&utm_medium=social&utm_campaign=ReviveOldPost
## 1 https://healthpayerintelligence.com/news/unitedhealthcare-combats-opioid-crisis-with-non-opioid-benefits
## Topic InternetSearch Contraindications RisksSideEffects
## 0 chiropractic benefits <NA> <NA> <NA>
## 1 chiropractic benefits <NA> <NA> <NA>
## body_length punct.
## 0 2497 2.3
## 1 4055 2.4
## Cleaned_text
## 0 ['chiropract', 'adjust', 'treatment', 'serv', 'need', 'million', 'peopl', 'around', 'world', 'adjust', 'offer', 'effect', 'noninvas', 'costeffect', 'solut', 'neck', 'back', 'pain', 'well', 'myriad', 'medic', 'issu', 'ever', 'stop', 'wonder', 'mani', 'us', 'suffer', 'neck', 'back', 'stiff', 'pain', 'apart', 'obviou', 'discomfort', 'simpl', 'daili', 'task', 'drive', 'car', 'cross', 'busi', 'street', 'pick', 'thing', 'floor', 'becom', 'challeng', 'individu', 'experienc', 'pain', 'anyon', 'experienc', 'pain', 'would', 'know', 'restrict', 'movement', 'debilit', 'unfortun', 'busi', 'world', 'doesnt', 'allow', 'us', 'stop', 'benefit', 'longterm', 'chiropract', 'care', 'includ', 'chiropractor', 'identifi', 'mechan', 'issu', 'caus', 'spinerel', 'pain', 'offer', 'seri', 'adjust', 'provid', 'near', 'immedi', 'relief', 'follow', 'appoint', 'patient', 'often', 'report', 'feel', 'symptom', 'notic', 'better', 'chiropractor', 'perform', 'adjust', 'help', 'restor', 'movement', 'joint', 'lock', 'becom', 'possibl', 'treatment', 'allow', 'muscl', 'surround', 'joint', 'relax', 'therebi', 'reduc', 'joint', 'stiff', 'mani', 'factor', 'affect', 'health', 'includ', 'exercis', 'pattern', 'nutrit', 'sleep', 'hered', 'environ', 'live', 'rather', 'treat', 'symptom', 'diseas', 'chiropract', 'care', 'focus', 'holist', 'approach', 'natur', 'maintain', 'health', 'resist', 'diseas', 'chiropract', 'adjust', 'help', 'restor', 'normal', 'function', 'movement', 'entir', 'bodi', 'mani', 'patient', 'report', 'improv', 'abil', 'move', 'effici', 'strength', 'mani', 'patient', 'find', 'delight', 'result', 'chiropract', 'adjust', 'old', 'chronic', 'injuri', 'whether', 'injuri', 'fact', 'new', 'old', 'chiropract', 'care', 'help', 'reduc', 'pain', 'restor', 'mobil', 'provid', 'quick', 'pain', 'relief', 'joint', 'bodi', 'care', 'help', 'maintain', 'better', 'overal', 'health', 'thu', 'faster', 'recoveri', 'time', 'ever', 'notic', 'pain', 'unabl', 'perform', 'regular', 'favorit', 'activ', 'put', 'strain', 'emot', 'mental', 'wellb', 'exampl', 'increas', 'stress', 'abl', 'properli', 'perform', 'paid', 'job', 'turn', 'neg', 'impact', 'physic', 'health', 'increas', 'heart', 'rate', 'blood', 'pressur', 'domino', 'effect', 'often', 'continu', 'sleep', 'becom', 'disturb', 'result', 'lethargi', 'tired', 'day', 'anyon', 'realli', 'feel', 'exercis', 'state', 'chiropract', 'care', 'natur', 'method', 'heal', 'bodi', 'commun', 'system', 'never', 'reli', 'use', 'pharmaceut', 'drug', 'invas', 'surgeri', 'doctor', 'chiropract', 'work', 'collabor', 'healthcar', 'profession', 'condit', 'requir', 'attent', 'anoth', 'healthcar', 'profess', 'recommend', 'referr', 'made', '']
## 1 ['', 'unitedhealthcar', 'combat', 'opioid', 'crisi', 'nonopioid', 'benefit', 'physic', 'therapi', 'chiropract', 'care', 'prevent', 'reduc', 'expens', 'invas', 'spinal', 'procedur', 'imag', 'surgeri', 'reduc', 'opioid', 'use', 'cut', 'cost', 'unitedhealthcar', 'opioid', 'physic', 'therapi', 'healthcar', 'spend', 'sourc', 'thinkstock', 'share', 'twitter', 'kelsey', 'waddil', 'octob', '29', '2019', 'unitedhealthcar', 'uhc', 'combat', 'opioid', 'epidem', 'high', 'healthcar', 'cost', 'new', 'physic', 'therapi', 'chiropract', 'care', 'benefit', 'prevent', 'delay', 'case', 'substitut', 'invas', 'spinal', 'procedur', 'million', 'american', 'experienc', 'low', 'back', 'pain', 'current', 'point', 'lifetim', 'believ', 'benefit', 'design', 'help', 'make', 'meaning', 'differ', 'improv', 'health', 'outcom', 'reduc', 'cost', 'said', 'ann', 'docimo', 'md', 'unitedhealthcar', 'chief', 'medic', 'offic', 'lower', 'back', 'pain', 'part', 'respons', 'sustain', 'opioid', 'epidem', 'also', 'increas', 'healthcar', 'cost', 'dig', 'deeper', 'major', 'payer', 'provid', 'substanc', 'abus', 'care', 'opioid', 'misus', 'opioid', 'overdos', 'fall', '2', '2017', '2018', 'cdc', 'report', 'physic', 'therapi', 'chiropract', 'back', 'care', 'cut', 'opioid', 'use', 'cost', 'although', 'opioid', 'overdos', 'fell', 'two', 'percent', '2017', '2018', 'legal', 'battl', 'aim', 'hold', 'pharmaceut', 'compani', 'account', 'end', 'sight', 'opioid', 'epidem', 'industri', 'profession', 'still', 'grappl', 'balanc', 'cut', 'opioid', 'prescript', 'work', 'reduc', 'patient', 'pain', 'common', 'condit', 'low', 'back', 'pain', 'bolster', 'epidem', 'presenc', 'clinician', 'still', 'prescrib', 'opioid', 'best', 'practic', 'recommend', 'accord', 'recent', 'optumlab', 'studi', '9', 'percent', 'patient', 'newli', 'diagnos', 'low', 'back', 'pain', 'prescrib', 'opioid', 'lower', 'back', 'pain', 'current', 'contribut', '52', 'percent', 'overal', 'opioid', 'prescript', 'rate', 'addit', 'boost', 'opioid', 'distribut', 'altern', 'invas', 'lower', 'back', 'pain', 'treatment', 'significantli', 'impact', 'healthcar', 'spend', 'new', 'inform', 'physic', 'therapi', 'chiropract', 'care', 'effect', 'lower', 'cost', 'altern', 'spinal', 'imag', 'surgeri', 'howev', 'payer', 'still', 'process', 'adopt', 'method', 'counteract', 'highcost', 'highrisk', 'potenti', 'use', 'opioid', 'treat', 'back', 'pain', 'uhc', 'creat', 'benefit', 'reli', 'medic', 'technolog', 'rather', 'physic', 'therapi', 'chiropract', 'care', 'benefit', 'allow', 'elig', 'employ', 'offer', 'physic', 'therapist', 'chiropractor', 'visit', 'outofpocket', 'cost', 'member', 'alreadi', 'receiv', 'physic', 'therapist', 'chiropract', 'care', 'benefit', 'uhc', 'employersponsor', 'health', 'plan', 'max', 'visit', 'receiv', 'addit', 'visit', 'benefit', 'howev', 'still', 'visit', 'use', 'choos', 'physic', 'therapi', 'chiropract', 'care', 'form', 'treatment', 'copay', 'deduct', 'visit', 'waiv', 'receiv', 'three', 'visit', 'cost', 'uhc', 'high', 'expect', 'fiscal', 'physic', 'impact', 'benefit', 'accord', 'uhc', 'analysi', 'health', 'payer', 'expect', '2021', 'opioid', 'use', 'decreas', '19', 'percent', 'spinal', 'imag', 'test', 'frequenc', 'spinal', 'surgeri', 'reduc', '22', 'percent', '21', 'percent', 'respect', 'addit', 'specif', 'goal', 'uhc', 'hope', 'see', 'decreas', 'overal', 'cost', 'spinal', 'care', 'optumlab', 'studi', 'demonstr', 'uhc', 'expect', 'without', 'preced', 'studi', 'look', 'correl', 'outofpocket', 'cost', 'patient', 'util', 'noninvas', 'treatment', 'research', 'discov', 'member', 'whose', 'copay', '30', 'littl', '30', 'percent', 'less', 'like', 'choos', 'physic', 'therapi', 'oppos', 'invas', 'treatment', 'american', 'journal', 'manag', 'care', 'studi', 'june', '2019', 'found', 'patient', 'high', 'deduct', 'typic', '1000', 'less', 'like', 'visit', 'physic', 'therapi', 'elig', 'employ', 'may', 'brand', 'new', 'renew', 'membership', 'must', 'fulli', 'insur', '51', 'employe', 'strong', 'benefit', 'current', 'avail', 'connecticut', 'florida', 'georgia', 'new', 'york', 'north', 'carolina', 'howev', 'uhc', 'plan', 'expand', 'benefit', '2020', '2021', 'end', 'expans', 'period', 'benefit', 'also', 'avail', 'selffund', 'employ', 'organ', 'employe', 'popul', '2', '50', 'benefit', 'span', 'ten', 'state', 'primarili', 'southeast', 'new', 'benefit', 'design', 'may', 'help', 'encourag', 'peopl', 'low', 'back', 'pain', 'get', 'right', 'care', 'right', 'time', 'right', 'set', 'help', 'expand', 'access', 'evidencebas', 'afford', 'treatment', 'said', 'docimo']
## Lemmatized
## 0 ['chiropractic', 'adjustment', 'treatment', 'serve', 'need', 'million', 'people', 'around', 'world', 'adjustment', 'offer', 'effective', 'noninvasive', 'costeffective', 'solution', 'neck', 'back', 'pain', 'well', 'myriad', 'medical', 'issue', 'ever', 'stopped', 'wonder', 'many', 'u', 'suffer', 'neck', 'back', 'stiffness', 'pain', 'apart', 'obvious', 'discomfort', 'simple', 'daily', 'task', 'driving', 'car', 'crossing', 'busy', 'street', 'picking', 'thing', 'floor', 'become', 'challenging', 'individual', 'experiencing', 'pain', 'anyone', 'experienced', 'pain', 'would', 'know', 'restricted', 'movement', 'debilitating', 'unfortunately', 'busy', 'world', 'doesnt', 'allow', 'u', 'stop', 'benefit', 'longterm', 'chiropractic', 'care', 'include', 'chiropractor', 'identify', 'mechanical', 'issue', 'cause', 'spinerelated', 'pain', 'offer', 'series', 'adjustment', 'provide', 'near', 'immediate', 'relief', 'following', 'appointment', 'patient', 'often', 'report', 'feeling', 'symptom', 'noticeably', 'better', 'chiropractor', 'performs', 'adjustment', 'help', 'restore', 'movement', 'joint', 'locked', 'becomes', 'possible', 'treatment', 'allows', 'muscle', 'surrounding', 'joint', 'relax', 'thereby', 'reducing', 'joint', 'stiffness', 'many', 'factor', 'affect', 'health', 'including', 'exercise', 'pattern', 'nutrition', 'sleep', 'heredity', 'environment', 'live', 'rather', 'treat', 'symptom', 'disease', 'chiropractic', 'care', 'focus', 'holistic', 'approach', 'naturally', 'maintain', 'health', 'resist', 'disease', 'chiropractic', 'adjustment', 'help', 'restore', 'normal', 'function', 'movement', 'entire', 'body', 'many', 'patient', 'report', 'improvement', 'ability', 'move', 'efficiency', 'strength', 'many', 'patient', 'find', 'delight', 'result', 'chiropractic', 'adjustment', 'old', 'chronic', 'injury', 'whether', 'injury', 'fact', 'new', 'old', 'chiropractic', 'care', 'help', 'reduce', 'pain', 'restore', 'mobility', 'provide', 'quick', 'pain', 'relief', 'joint', 'body', 'care', 'help', 'maintain', 'better', 'overall', 'health', 'thus', 'faster', 'recovery', 'time', 'ever', 'noticed', 'pain', 'unable', 'perform', 'regular', 'favorite', 'activity', 'put', 'strain', 'emotional', 'mental', 'wellbeing', 'example', 'increased', 'stress', 'able', 'properly', 'perform', 'paid', 'job', 'turn', 'negative', 'impact', 'physical', 'health', 'increase', 'heart', 'rate', 'blood', 'pressure', 'domino', 'effect', 'often', 'continues', 'sleep', 'becoming', 'disturbed', 'resulting', 'lethargy', 'tiredness', 'day', 'anyone', 'really', 'feel', 'exercising', 'state', 'chiropractic', 'care', 'natural', 'method', 'healing', 'body', 'communication', 'system', 'never', 'relies', 'use', 'pharmaceutical', 'drug', 'invasive', 'surgery', 'doctor', 'chiropractic', 'work', 'collaboratively', 'healthcare', 'professional', 'condition', 'require', 'attention', 'another', 'healthcare', 'profession', 'recommendation', 'referral', 'made', '']
## 1 ['', 'unitedhealthcare', 'combat', 'opioid', 'crisis', 'nonopioid', 'benefit', 'physical', 'therapy', 'chiropractic', 'care', 'prevent', 'reduce', 'expensive', 'invasive', 'spinal', 'procedure', 'imaging', 'surgery', 'reduce', 'opioid', 'use', 'cut', 'cost', 'unitedhealthcare', 'opioid', 'physical', 'therapy', 'healthcare', 'spending', 'source', 'thinkstock', 'share', 'twitter', 'kelsey', 'waddill', 'october', '29', '2019', 'unitedhealthcare', 'uhc', 'combatting', 'opioid', 'epidemic', 'high', 'healthcare', 'cost', 'new', 'physical', 'therapy', 'chiropractic', 'care', 'benefit', 'prevent', 'delay', 'case', 'substitute', 'invasive', 'spinal', 'procedure', 'million', 'american', 'experiencing', 'low', 'back', 'pain', 'currently', 'point', 'lifetime', 'believe', 'benefit', 'design', 'help', 'make', 'meaningful', 'difference', 'improving', 'health', 'outcome', 'reducing', 'cost', 'said', 'anne', 'docimo', 'md', 'unitedhealthcare', 'chief', 'medical', 'officer', 'lower', 'back', 'pain', 'part', 'responsible', 'sustaining', 'opioid', 'epidemic', 'also', 'increase', 'healthcare', 'cost', 'dig', 'deeper', 'major', 'payer', 'provide', 'substance', 'abuse', 'care', 'opioid', 'misuse', 'opioid', 'overdoses', 'fall', '2', '2017', '2018', 'cdc', 'report', 'physical', 'therapy', 'chiropractic', 'back', 'care', 'cut', 'opioid', 'use', 'cost', 'although', 'opioid', 'overdoses', 'fell', 'two', 'percent', '2017', '2018', 'legal', 'battle', 'aim', 'hold', 'pharmaceutical', 'company', 'accountable', 'end', 'sight', 'opioid', 'epidemic', 'industry', 'professional', 'still', 'grappling', 'balance', 'cutting', 'opioid', 'prescription', 'working', 'reduce', 'patient', 'pain', 'common', 'condition', 'low', 'back', 'pain', 'bolster', 'epidemic', 'presence', 'clinician', 'still', 'prescribing', 'opioids', 'best', 'practice', 'recommendation', 'according', 'recent', 'optumlabs', 'study', '9', 'percent', 'patient', 'newly', 'diagnosed', 'low', 'back', 'pain', 'prescribed', 'opioids', 'lower', 'back', 'pain', 'currently', 'contributes', '52', 'percent', 'overall', 'opioid', 'prescription', 'rate', 'addition', 'boosting', 'opioids', 'distribution', 'alternative', 'invasive', 'lower', 'back', 'pain', 'treatment', 'significantly', 'impact', 'healthcare', 'spending', 'new', 'information', 'physical', 'therapy', 'chiropractic', 'care', 'effective', 'lower', 'cost', 'alternative', 'spinal', 'imaging', 'surgery', 'however', 'payer', 'still', 'process', 'adopting', 'method', 'counteract', 'highcost', 'highrisk', 'potential', 'using', 'opioids', 'treat', 'back', 'pain', 'uhc', 'created', 'benefit', 'rely', 'medication', 'technology', 'rather', 'physical', 'therapy', 'chiropractic', 'care', 'benefit', 'allows', 'eligible', 'employer', 'offer', 'physical', 'therapist', 'chiropractor', 'visit', 'outofpocket', 'cost', 'member', 'already', 'receive', 'physical', 'therapist', 'chiropractic', 'care', 'benefit', 'uhcs', 'employersponsored', 'health', 'plan', 'maxed', 'visit', 'receive', 'additional', 'visit', 'benefit', 'however', 'still', 'visit', 'use', 'choose', 'physical', 'therapy', 'chiropractic', 'care', 'form', 'treatment', 'copay', 'deductible', 'visit', 'waived', 'receive', 'three', 'visit', 'cost', 'uhc', 'high', 'expectation', 'fiscal', 'physical', 'impact', 'benefit', 'according', 'uhcs', 'analysis', 'health', 'payer', 'expects', '2021', 'opioid', 'use', 'decrease', '19', 'percent', 'spinal', 'imaging', 'test', 'frequency', 'spinal', 'surgery', 'reduced', '22', 'percent', '21', 'percent', 'respectively', 'addition', 'specific', 'goal', 'uhc', 'hope', 'see', 'decrease', 'overall', 'cost', 'spinal', 'care', 'optumlabs', 'study', 'demonstrated', 'uhcs', 'expectation', 'without', 'precedent', 'study', 'looked', 'correlation', 'outofpocket', 'cost', 'patient', 'utilization', 'noninvasive', 'treatment', 'researcher', 'discovered', 'member', 'whose', 'copay', '30', 'little', '30', 'percent', 'le', 'likely', 'choose', 'physical', 'therapy', 'opposed', 'invasive', 'treatment', 'american', 'journal', 'managed', 'care', 'study', 'june', '2019', 'found', 'patient', 'high', 'deductible', 'typically', '1000', 'le', 'likely', 'visit', 'physical', 'therapy', 'eligible', 'employer', 'may', 'brand', 'new', 'renewing', 'membership', 'must', 'fully', 'insured', '51', 'employee', 'strong', 'benefit', 'currently', 'available', 'connecticut', 'florida', 'georgia', 'new', 'york', 'north', 'carolina', 'however', 'uhc', 'plan', 'expand', 'benefit', '2020', '2021', 'end', 'expansion', 'period', 'benefit', 'also', 'available', 'selffunded', 'employer', 'organization', 'employee', 'population', '2', '50', 'benefit', 'span', 'ten', 'state', 'primarily', 'southeast', 'new', 'benefit', 'design', 'may', 'help', 'encourage', 'people', 'low', 'back', 'pain', 'get', 'right', 'care', 'right', 'time', 'right', 'setting', 'helping', 'expand', 'access', 'evidencebased', 'affordable', 'treatment', 'said', 'docimo']from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(data[['Document','body_length','punct%','Cleaned_text','Lemmatized']],data['Topic'],test_size=0.20)from sklearn.feature_extraction.text import CountVectorizercount_vect=CountVectorizer(analyzer=clean_text)
count_vect_fit=count_vect.fit(X_train['Document'])
count_train=count_vect_fit.transform(X_train['Document'])
count_test=count_vect_fit.transform(X_test['Document'])len(count_vect_fit.get_feature_names())## 3554count_vect_fit.get_feature_names()[200:350]## ['adolesc', 'adopt', 'ador', 'adult', 'adults1', 'advanc', 'advantag', 'advers', 'advertis', 'advic', 'advis', 'advisori', 'advoc', 'aerob', 'affair', 'affect', 'affili', 'afford', 'afterhour', 'afterward', 'agakian', 'age', 'agenc', 'agephys', 'agerel', 'aggress', 'agil', 'ago', 'agre', 'ahead', 'aid', 'ailment', 'aim', 'air', 'airport', 'aka', 'alchemist', 'alcohol', 'alert', 'align', 'alik', 'allerg', 'allergi', 'allevi', 'allianc', 'allison', 'allow', 'almost', 'alon', 'along', 'alongsid', 'alreadi', 'alright', 'also', 'alter', 'altern', 'alternatingli', 'although', 'altogeth', 'alway', 'alzheim', 'amateur', 'amaz', 'amazon', 'amazoncom', 'ambul', 'amelior', 'america', 'american', 'ami', 'among', 'amount', 'amplitud', 'amta', 'analysi', 'analyz', 'anatomi', 'ancient', 'andor', 'anecdot', 'anemia', 'anesthet', 'angel', 'anger', 'angl', 'angri', 'anim', 'ankl', 'ann', 'annal', 'anni', 'announc', 'annual', 'anoth', 'answer', 'antibiot', 'anticip', 'antiinflammatori', 'antivir', 'anxieti', 'anxietydepress', 'anxietyfre', 'anxiou', 'anyon', 'anyth', 'anywher', 'apa', 'apart', 'appeal', 'appear', 'appendix', 'appetit', 'appli', 'applianc', 'applic', 'appoint', 'approach', 'appropri', 'approv', 'approxim', 'apr', 'april', 'apta', 'arab', 'archiv', 'area', 'arent', 'aris', 'arm', 'armonk', 'aromatherapi', 'around', 'array', 'arriv', 'art', 'arteri', 'arthrit', 'arthriti', 'articl', 'articles2', 'asid', 'ask', 'aspect', 'aspir', 'aspirin', 'assert', 'assess', 'assign', 'assist', 'associ']count_train_vect=pd.concat([X_train[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(count_train.toarray())],axis=1)
count_test_vect=pd.concat([X_test[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(count_test.toarray())],axis=1)count_train_vect.head()## Document body_length ... 3552 3553
## 0 Learn All About Cupping Therapy And The Many ... 2204 ... 0 0
## 1 \r\nThe Challenges and Benefits of Treatment f... 4558 ... 0 0
## 2 Goodnight everyone, say it back or Ill massag... 66 ... 0 0
## 3 I adore men who massage me all over my body 34 ... 0 0
## 4 How Physical Therapy Can Help Your Recovery\r... 3563 ... 0 0
##
## [5 rows x 3559 columns]from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import precision_recall_fscore_support as score
import timerf=RandomForestClassifier(n_estimators=150, max_depth=None, n_jobs=-1)
start=time.time()
rf_model=rf.fit(count_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=rf_model.predict(count_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 3 chiropractic benefits chiropractic benefits
## 78 Not Professional Not Professional
## 2 chiropractic benefits chiropractic benefits
## 35 cupping benefits cupping benefits
## 62 cold stone benefits cold stone benefits
## 51 massage gun benefits massage gun benefits
## 68 Not Professional Not Professional
## 80 Not Professional Not Professional
## 4 massage benefits massage benefits
## 8 Not Professional massage benefits
## 69 Not Professional Not Professional
## 71 Not Professional Not Professional
## 20 physical therapy benefits physical therapy benefits
## 5 massage benefits massage benefits
## 38 cupping benefits cupping benefits
## 26 mental health services benefits mental health services benefits
## 50 massage gun benefits massage gun benefits
## 70 Not Professional Not Professionalfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.9444444444444444print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[6 0 0 0 0 0 0 0]
## [0 2 0 0 0 0 0 0]
## [0 0 1 0 0 0 0 0]
## [0 0 0 2 0 0 0 0]
## [1 0 0 0 2 0 0 0]
## [0 0 0 0 0 2 0 0]
## [0 0 0 0 0 0 1 0]
## [0 0 0 0 0 0 0 1]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 0.86 1.00 0.92 6
## chiropractic benefits 1.00 1.00 1.00 2
## cold stone benefits 1.00 1.00 1.00 1
## cupping benefits 1.00 1.00 1.00 2
## massage benefits 1.00 0.67 0.80 3
## massage gun benefits 1.00 1.00 1.00 2
## mental health services benefits 1.00 1.00 1.00 1
## physical therapy benefits 1.00 1.00 1.00 1
##
## accuracy 0.94 18
## macro avg 0.98 0.96 0.97 18
## weighted avg 0.95 0.94 0.94 18gb=GradientBoostingClassifier(n_estimators=150,max_depth=11)
start=time.time()
gb_model=gb.fit(count_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=gb_model.predict(count_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 3 chiropractic benefits chiropractic benefits
## 78 Not Professional Not Professional
## 2 chiropractic benefits chiropractic benefits
## 35 cupping benefits cupping benefits
## 62 cold stone benefits cold stone benefits
## 51 massage gun benefits massage gun benefits
## 68 Not Professional Not Professional
## 80 Not Professional Not Professional
## 4 massage benefits massage benefits
## 8 Not Professional massage benefits
## 69 Not Professional Not Professional
## 71 Not Professional Not Professional
## 20 physical therapy benefits physical therapy benefits
## 5 massage benefits massage benefits
## 38 cupping benefits cupping benefits
## 26 mental health services benefits mental health services benefits
## 50 massage gun benefits massage gun benefits
## 70 Not Professional Not Professionalfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.9444444444444444print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[6 0 0 0 0 0 0 0]
## [0 2 0 0 0 0 0 0]
## [0 0 1 0 0 0 0 0]
## [0 0 0 2 0 0 0 0]
## [1 0 0 0 2 0 0 0]
## [0 0 0 0 0 2 0 0]
## [0 0 0 0 0 0 1 0]
## [0 0 0 0 0 0 0 1]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 0.86 1.00 0.92 6
## chiropractic benefits 1.00 1.00 1.00 2
## cold stone benefits 1.00 1.00 1.00 1
## cupping benefits 1.00 1.00 1.00 2
## massage benefits 1.00 0.67 0.80 3
## massage gun benefits 1.00 1.00 1.00 2
## mental health services benefits 1.00 1.00 1.00 1
## physical therapy benefits 1.00 1.00 1.00 1
##
## accuracy 0.94 18
## macro avg 0.98 0.96 0.97 18
## weighted avg 0.95 0.94 0.94 18TF-IDF RFC and GBC
stopwords = nltk.corpus.stopwords.words('english')
ps=nltk.PorterStemmer()
wn=nltk.WordNetLemmatizer()
data=pd.read_csv("benefitsContraindications3.csv", encoding='unicode_escape')
data.columns=['Document','Source','Topic','InternetSearch','Contraindications','RisksSideEffects']def count_punct(text):
count=sum([1 for char in text if char in string.punctuation])
return round(count/(len(text)-text.count(" ")),3)*100data['body_length']=data['Document'].apply(lambda x: len(x)-x.count(" "))
data['punct%']= data['Document'].apply(lambda x: count_punct(x))
def clean_text(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=[ps.stem(word) for word in tokens if word not in stopwords]#these are lists for the count vectorizer
return text
def lemmatize(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=[wn.lemmatize(word) for word in tokens if word not in stopwords]#these are lists for the count vectorizer
return text
data['Cleaned_text']=data['Document'].apply(lambda x: clean_text(x))
data['Lemmatized']=data['Document'].apply(lambda x: lemmatize(x))
data.head()## Document ... Lemmatized
## 0 Chiropractic adjustments and treatments serve ... ... [chiropractic, adjustment, treatment, serve, n...
## 1 \r\nUnitedHealthcare Combats Opioid Crisis wit... ... [, unitedhealthcare, combat, opioid, crisis, n...
## 2 The Safety of Chiropractic Adjustments\r\nBy ... ... [, safety, chiropractic, adjustment, lana, bar...
## 3 Advanced Chiropractic Relief: 8 Key Benefits o... ... [advanced, chiropractic, relief, 8, key, benef...
## 4 Heading to the spa can be a pampering treat, b... ... [heading, spa, pampering, treat, also, huge, b...
##
## [5 rows x 10 columns]
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(data[['Document','body_length','punct%','Cleaned_text','Lemmatized']],data['Topic'],test_size=0.20)
from sklearn.feature_extraction.text import TfidfVectorizer
tfidf_vect=TfidfVectorizer(analyzer=clean_text)
tfidf_vect_fit=tfidf_vect.fit(X_train['Document'])
tfidf_train=tfidf_vect_fit.transform(X_train['Document'])
tfidf_test=tfidf_vect_fit.transform(X_test['Document'])len(tfidf_vect_fit.get_feature_names())## 3664tfidf_vect_fit.get_feature_names()[200:350]## ['adminmcb', 'admiss', 'admit', 'adolesc', 'adopt', 'ador', 'adult', 'adulteri', 'adults1', 'advanc', 'advantag', 'advers', 'advertis', 'advic', 'advis', 'advisori', 'advoc', 'aerob', 'affair', 'affect', 'affili', 'affirm', 'afford', 'afraid', 'afterhour', 'afterward', 'age', 'agenc', 'agerel', 'aggress', 'agil', 'ago', 'agre', 'ahead', 'aid', 'ailment', 'aim', 'air', 'airport', 'ajman', 'al', 'alcohol', 'alcoholsoak', 'alert', 'align', 'alik', 'allerg', 'allergi', 'allevi', 'allianc', 'allison', 'allow', 'almost', 'alon', 'along', 'alongsid', 'alreadi', 'alright', 'also', 'alter', 'altern', 'although', 'altogeth', 'alway', 'alzheim', 'amateur', 'amaz', 'amazon', 'amazoncom', 'ambul', 'amelior', 'america', 'american', 'ami', 'among', 'amount', 'amplitud', 'amta', 'analges', 'analysi', 'analyz', 'anatomi', 'ancient', 'andor', 'anecdot', 'anemia', 'anesthet', 'angel', 'anger', 'angl', 'angri', 'anhedonia', 'aniston', 'ankl', 'ann', 'annal', 'anni', 'announc', 'annual', 'anosognosia', 'anoth', 'answer', 'antibiot', 'anticip', 'antidepress', 'antiinflammatori', 'antivir', 'anxieti', 'anxietyfre', 'anxiou', 'anyon', 'anyth', 'anywher', 'apa', 'apart', 'appeal', 'appear', 'appendix', 'appetit', 'appli', 'applianc', 'applic', 'appoint', 'approach', 'appropri', 'approv', 'approxim', 'apr', 'april', 'apta', 'archiv', 'area', 'areaswer', 'arent', 'aris', 'arm', 'aromatherapi', 'around', 'arquett', 'array', 'arriv', 'arrow', 'art', 'arteri', 'arthriti', 'articl', 'articles2', 'ascertain', 'asid', 'ask']tfidf_train_vect=pd.concat([X_train[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(tfidf_train.toarray())],axis=1)
tfidf_test_vect=pd.concat([X_test[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(tfidf_test.toarray())],axis=1)tfidf_train_vect.head()## Document body_length ... 3662 3663
## 0 Goodnight everyone, say it back or Ill massag... 66 ... 0.0 0.0
## 1 Is It an Emergency?\r\n\r\nConditions we treat... 721 ... 0.0 0.0
## 2 The Role of Physical Therapy\r\n\r\nThe role o... 4704 ... 0.0 0.0
## 3 \r\nThe Top 7 Benefits of Physical Therapy\r\n... 2947 ... 0.0 0.0
## 4 \r\nHow to know where to go for sudden health ... 5420 ... 0.0 0.0
##
## [5 rows x 3669 columns]from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import precision_recall_fscore_support as score
import time
np.random.seed(45678)rf=RandomForestClassifier(n_estimators=150, max_depth=None, n_jobs=-1)
start=time.time()
rf_model=rf.fit(tfidf_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=rf_model.predict(tfidf_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 79 massage benefits Not Professional
## 20 physical therapy benefits physical therapy benefits
## 38 cupping benefits cupping benefits
## 60 cold stone benefits cold stone benefits
## 63 cold stone benefits cold stone benefits
## 73 Not Professional Not Professional
## 17 physical therapy benefits physical therapy benefits
## 59 cold stone benefits cold stone benefits
## 76 Not Professional Not Professional
## 15 massage benefits massage benefits
## 19 massage benefits physical therapy benefits
## 18 massage benefits physical therapy benefits
## 57 cold stone benefits cold stone benefits
## 67 Not Professional Not Professional
## 7 massage benefits massage benefits
## 44 cupping benefits cupping benefits
## 31 Not Professional chiropractic benefits
## 46 massage gun benefits massage gun benefitsfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.7777777777777778print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[3 0 0 0 1 0 0]
## [1 0 0 0 0 0 0]
## [0 0 4 0 0 0 0]
## [0 0 0 2 0 0 0]
## [0 0 0 0 2 0 0]
## [0 0 0 0 0 1 0]
## [0 0 0 0 2 0 2]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 0.75 0.75 0.75 4
## chiropractic benefits 0.00 0.00 0.00 1
## cold stone benefits 1.00 1.00 1.00 4
## cupping benefits 1.00 1.00 1.00 2
## massage benefits 0.40 1.00 0.57 2
## massage gun benefits 1.00 1.00 1.00 1
## physical therapy benefits 1.00 0.50 0.67 4
##
## accuracy 0.78 18
## macro avg 0.74 0.75 0.71 18
## weighted avg 0.82 0.78 0.77 18
##
##
## C:\Users\m\Anaconda2\envs\python36\lib\site-packages\sklearn\metrics\classification.py:1437: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.
## 'precision', 'predicted', average, warn_for)gb=GradientBoostingClassifier(n_estimators=150,max_depth=11)
start=time.time()
gb_model=gb.fit(tfidf_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=gb_model.predict(tfidf_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 79 massage benefits Not Professional
## 20 physical therapy benefits physical therapy benefits
## 38 cupping benefits cupping benefits
## 60 cold stone benefits cold stone benefits
## 63 cold stone benefits cold stone benefits
## 73 Not Professional Not Professional
## 17 physical therapy benefits physical therapy benefits
## 59 cold stone benefits cold stone benefits
## 76 Not Professional Not Professional
## 15 massage benefits massage benefits
## 19 massage benefits physical therapy benefits
## 18 physical therapy benefits physical therapy benefits
## 57 cold stone benefits cold stone benefits
## 67 Not Professional Not Professional
## 7 massage benefits massage benefits
## 44 cupping benefits cupping benefits
## 31 chiropractic benefits chiropractic benefits
## 46 massage gun benefits massage gun benefitsfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.8888888888888888print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[3 0 0 0 1 0 0]
## [0 1 0 0 0 0 0]
## [0 0 4 0 0 0 0]
## [0 0 0 2 0 0 0]
## [0 0 0 0 2 0 0]
## [0 0 0 0 0 1 0]
## [0 0 0 0 1 0 3]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 1.00 0.75 0.86 4
## chiropractic benefits 1.00 1.00 1.00 1
## cold stone benefits 1.00 1.00 1.00 4
## cupping benefits 1.00 1.00 1.00 2
## massage benefits 0.50 1.00 0.67 2
## massage gun benefits 1.00 1.00 1.00 1
## physical therapy benefits 1.00 0.75 0.86 4
##
## accuracy 0.89 18
## macro avg 0.93 0.93 0.91 18
## weighted avg 0.94 0.89 0.90 18N-Grams Vectorization for RFC and GBC
stopwords = nltk.corpus.stopwords.words('english')
ps=nltk.PorterStemmer()
wn=nltk.WordNetLemmatizer()
data=pd.read_csv("benefitsContraindications3.csv", encoding='unicode_escape')
data.columns=['Document','Source','Topic','InternetSearch','Contraindications','RisksSideEffects']def count_punct(text):
count=sum([1 for char in text if char in string.punctuation])
return round(count/(len(text)-text.count(" ")),3)*100data['body_length']=data['Document'].apply(lambda x: len(x)-x.count(" "))
data['punct%']= data['Document'].apply(lambda x: count_punct(x))def clean_text(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+',text)
text=" ".join([ps.stem(word) for word in tokens if word not in stopwords])#unlisted with N-grams vectorization
return text
def lemmatize(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=" ".join([wn.lemmatize(word) for word in tokens if word not in stopwords])#unlisted with N-grams vectorization
#text=[wn.lemmatize(word) for word in tokens if word not in stopwords]#when using count Vectorization its a list
#or else single letters returned.
return text data['Cleaned_text']=data['Document'].apply(lambda x: clean_text(x))
data['Lemmatized']=data['Document'].apply(lambda x: lemmatize(x))
data.head()## Document ... Lemmatized
## 0 Chiropractic adjustments and treatments serve ... ... chiropractic adjustment treatment serve need m...
## 1 \r\nUnitedHealthcare Combats Opioid Crisis wit... ... unitedhealthcare combat opioid crisis nonopio...
## 2 The Safety of Chiropractic Adjustments\r\nBy ... ... safety chiropractic adjustment lana barhum me...
## 3 Advanced Chiropractic Relief: 8 Key Benefits o... ... advanced chiropractic relief 8 key benefit chi...
## 4 Heading to the spa can be a pampering treat, b... ... heading spa pampering treat also huge boost he...
##
## [5 rows x 10 columns]from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(data[['Document','body_length','punct%','Cleaned_text','Lemmatized']],data['Topic'],test_size=0.20)from sklearn.feature_extraction.text import CountVectorizern_gram_vect=CountVectorizer(ngram_range=(1,4))type(X_train['Cleaned_text'])## <class 'pandas.core.series.Series'>X_train['Cleaned_text'].head()## 69 avail good massag plu servic sex top nice romant
## 68 good morn ajman spend day beauti girl monday r...
## 56 cold stone massag benefici adminmcb januari 21...
## 0 chiropract adjust treatment serv need million ...
## 31 21 benefit chiropract adjust woman chiropract...
## Name: Cleaned_text, dtype: objectX_train['Lemmatized'].head()## 69 available good massage plus service sex top ni...
## 68 good morning ajman spend day beautiful girl mo...
## 56 cold stone massage beneficial adminmcb january...
## 0 chiropractic adjustment treatment serve need m...
## 31 21 benefit chiropractic adjustment woman chir...
## Name: Lemmatized, dtype: objectn_gram_vect_fit=n_gram_vect.fit(X_train['Cleaned_text'])
n_gram_train=n_gram_vect_fit.transform(X_train['Cleaned_text'])
n_gram_test=n_gram_vect_fit.transform(X_test['Cleaned_text'])len(n_gram_vect_fit.get_feature_names())## 71655print(n_gram_vect_fit.get_feature_names()[200:500])## ['18999 amazoncom damke profession', '18999 damke', '18999 damke current', '18999 damke current sale', '18h', '18h lmassagebodytobodi', '18h lmassagebodytobodi massag', '18h lmassagebodytobodi massag massag', '19', '19 percent', '19 percent spinal', '19 percent spinal imag', '192', '192 peopl', '192 peopl backrel', '192 peopl backrel pain', '1950', '1950 across', '1950 across hospit', '1950 across hospit china', '1950 cup', '1950 cup also', '1950 cup also practic', '1996', '1996 studi', '1996 studi rand', '1996 studi rand corpor', '19999', '19999 amazoncom', '19999 amazoncom 4000', '19999 amazoncom 4000 amazon', '20', '20 2015', '20 2015 therapi', '20 2015 therapi comment', '20 differ', '20 differ adjust', '20 differ adjust speed', '20 minut', '20 minut depend', '20 minut depend natur', '20 minut long', '20 minut long say', '20 year', '20 year without', '20 year without advers', '2000', '2000 3000', '2000 3000 hour', '2000 3000 hour supervis', '2001', '2001 overview', '2001 overview score', '2001 overview score phq9', '2001 studi', '2001 studi publish', '2001 studi publish canadian', '2003', '2003 evalu', '2003 evalu 183', '2003 evalu 183 patient', '2003 score', '2003 score might', '2003 score might proceed', '2005', '2005 studi', '2005 studi publish', '2005 studi publish intern', '2007', '2007 accord', '2007 accord nation', '2007 accord nation health', '2007 articl', '2007 articl chicago', '2007 articl chicago tribun', '2007 like', '2007 like seek', '2007 like seek coach', '2007 stress', '2007 stress also', '2007 stress also lead', '2008', '2008 center', '2008 center diseas', '2008 center diseas control', '2008 grown', '2008 grown becom', '2008 grown becom favorit', '2008 research', '2008 research evalu', '2008 research evalu chiropract', '2009', '2009 diseas', '2009 diseas link', '2009 diseas link inflamm', '2009 found', '2009 found reflect', '2009 found reflect journal', '2009 reduct', '2009 reduct chronic', '2009 reduct chronic pain', '2010', '2010 studi', '2010 studi publish', '2010 studi publish journal', '2011', '2011 cnnmoneycom', '2011 cnnmoneycom gave', '2011 cnnmoneycom gave physic', '2011 number', '2011 number adult', '2011 number adult us', '2011 review', '2011 review 26', '2011 review 26 clinic', '2011 studi', '2011 studi publish', '2011 studi publish annal', '2012', '2012 report', '2012 report publish', '2012 report publish plo', '2012 review', '2012 review studiestrust', '2012 review studiestrust sourc', '2012 well', '2012 well benefit', '2012 well benefit societi', '2013', '2013 2011', '2013 2011 cnnmoneycom', '2013 2011 cnnmoneycom gave', '2013 illustr', '2013 illustr benefit', '2013 illustr benefit journal', '2014', '2014 2024a', '2014 2024a much', '2014 2024a much quicker', '2014 studi', '2014 studi suggest', '2014 studi suggest vibrat', '2015', '2015 journal', '2015 journal tradit', '2015 journal tradit complementari', '2015 report', '2015 report publish', '2015 report publish journal', '2015 review', '2015 review evid', '2015 review evid found', '2015 therapi', '2015 therapi comment', '2015 therapi comment physic', '2015patricia', '2015patricia mayrhof', '2015patricia mayrhof spatechniqu', '2015patricia mayrhof spatechniqu articles2', '2016', '2016 930', '2016 930 robert', '2016 930 robert shmerl', '2016 cold', '2016 cold stone', '2016 cold stone massag', '2016 michael', '2016 michael phelp', '2016 michael phelp perman', '2016 mr', '2016 mr castaneda', '2016 mr castaneda written', '2016 studi', '2016 studi publish', '2016 studi publish journal', '2016 summer', '2016 summer olymp', '2016 summer olymp share', '2016 use', '2016 use cup', '2016 use cup easili', '2017', '2017 2018', '2017 2018 cdc', '2017 2018 cdc report', '2017 2018 legal', '2017 2018 legal battl', '2017 chiropractor', '2017 chiropractor tout', '2017 chiropractor tout treatment', '2017 nba', '2017 nba final', '2017 nba final irv', '2017 studi', '2017 studi found', '2017 studi found structur', '2018', '2018 84000', '2018 84000 topic', '2018 84000 topic physic', '2018 accord', '2018 accord american', '2018 accord american massag', '2018 cdc', '2018 cdc report', '2018 cdc report physic', '2018 find', '2018 find help', '2018 find help comment', '2018 galluppalm', '2018 galluppalm colleg', '2018 galluppalm colleg chiropract', '2018 getti', '2018 getti imag', '2018 getti imag cup', '2018 individu', '2018 individu salari', '2018 individu salari vari', '2018 largest', '2018 largest percentag', '2018 largest percentag work', '2018 legal', '2018 legal battl', '2018 legal battl aim', '2018 studi', '2018 studi led', '2018 studi led dr', '2019', '2019 1011', '2019 1011 articl', '2019 1011 articl base', '2019 1122', '2019 1122 share', '2019 1122 share articl', '2019 342', '2019 342 pm', '2019 342 pm edt', '2019 earli', '2019 earli 2020', '2019 earli 2020 mani', '2019 found', '2019 found patient', '2019 found patient high', '2019 massag', '2019 massag gun', '2019 massag gun one', '2019 physic', '2019 physic therapist', '2019 physic therapist stretch', '2019 puhhhagetti', '2019 puhhhagetti imag', '2019 puhhhagetti imag come', '2019 rich', '2019 rich barnesgetti', '2019 rich barnesgetti imag', '2019 stori', '2019 stori previous', '2019 stori previous publish', '2019 thai', '2019 thai massag', '2019 thai massag full', '2019 unitedhealthcar', '2019 unitedhealthcar uhc', '2019 unitedhealthcar uhc combat', '2020', '2020 2021', '2020 2021 end', '2020 2021 end expans', '2020 beyond', '2020 beyond peopl', '2020 beyond peopl say', '2020 click', '2020 click read', '2020 click read user', '2020 depress', '2020 depress man', '2020 depress man lie', '2020 karen', '2020 karen green', '2020 karen green last', '2020 mani', '2020 mani peopl', '2020 mani peopl start', '2020 may', '2020 may look', '2020 may look like', '2020 orthoped', '2020 orthoped orthoped', '2020 orthoped orthoped home', '2021', '2021 end', '2021 end expans', '2021 end expans period', '2021 opioid', '2021 opioid use', '2021 opioid use decreas', '2024', '2024 love', '2024 love job', '2024 love job help']n_gram_train_df=pd.concat([X_train[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(n_gram_train.toarray())],axis=1)
n_gram_test_df=pd.concat([X_test[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(n_gram_test.toarray())],axis=1)n_gram_train_df.head()## Document body_length ... 71653 71654
## 0 Available now.. good massage plus service sex.... 68 ... 0 0
## 1 Good morning Ajman.\r\nSpend your day with bea... 157 ... 0 0
## 2 Cold Stone Massage is Beneficial, Too - by adm... 1606 ... 0 0
## 3 Chiropractic adjustments and treatments serve ... 2497 ... 0 0
## 4 \r\n21 Benefits of Chiropractic Adjustments\r\... 1546 ... 0 0
##
## [5 rows x 71660 columns]from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import precision_recall_fscore_support as score
import timerf=RandomForestClassifier(n_estimators=150, max_depth=None, n_jobs=-1)
start=time.time()
rf_model=rf.fit(n_gram_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=rf_model.predict(n_gram_test)
end=time.time()
pred_time=(end-start)
prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 3 massage benefits chiropractic benefits
## 41 cupping benefits cupping benefits
## 62 cold stone benefits cold stone benefits
## 48 Not Professional massage gun benefits
## 43 Not Professional cupping benefits
## 70 Not Professional Not Professional
## 25 Not Professional mental health services benefits
## 75 Not Professional Not Professional
## 21 physical therapy benefits physical therapy benefits
## 15 Not Professional massage benefits
## 77 Not Professional Not Professional
## 55 massage benefits cold stone benefits
## 11 Not Professional massage benefits
## 36 Not Professional cupping benefits
## 33 Not Professional chiropractic benefits
## 32 Not Professional chiropractic benefits
## 18 Not Professional physical therapy benefits
## 84 Not Professional ERfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.3333333333333333print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[0 1 0 0 0 0 0 0 0]
## [0 3 0 0 0 0 0 0 0]
## [0 2 0 0 0 1 0 0 0]
## [0 0 0 1 0 1 0 0 0]
## [0 2 0 0 1 0 0 0 0]
## [0 2 0 0 0 0 0 0 0]
## [0 1 0 0 0 0 0 0 0]
## [0 1 0 0 0 0 0 0 0]
## [0 1 0 0 0 0 0 0 1]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## ER 0.00 0.00 0.00 1
## Not Professional 0.23 1.00 0.38 3
## chiropractic benefits 0.00 0.00 0.00 3
## cold stone benefits 1.00 0.50 0.67 2
## cupping benefits 1.00 0.33 0.50 3
## massage benefits 0.00 0.00 0.00 2
## massage gun benefits 0.00 0.00 0.00 1
## mental health services benefits 0.00 0.00 0.00 1
## physical therapy benefits 1.00 0.50 0.67 2
##
## accuracy 0.33 18
## macro avg 0.36 0.26 0.25 18
## weighted avg 0.43 0.33 0.29 18
##
##
## C:\Users\m\Anaconda2\envs\python36\lib\site-packages\sklearn\metrics\classification.py:1437: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.
## 'precision', 'predicted', average, warn_for)gb=GradientBoostingClassifier(n_estimators=150,max_depth=11)
start=time.time()
gb_model=gb.fit(n_gram_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=gb_model.predict(n_gram_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 3 chiropractic benefits chiropractic benefits
## 41 cupping benefits cupping benefits
## 62 cold stone benefits cold stone benefits
## 48 massage gun benefits massage gun benefits
## 43 cupping benefits cupping benefits
## 70 Not Professional Not Professional
## 25 mental health services benefits mental health services benefits
## 75 Not Professional Not Professional
## 21 physical therapy benefits physical therapy benefits
## 15 massage benefits massage benefits
## 77 Not Professional Not Professional
## 55 massage benefits cold stone benefits
## 11 massage benefits massage benefits
## 36 cupping benefits cupping benefits
## 33 chiropractic benefits chiropractic benefits
## 32 chiropractic benefits chiropractic benefits
## 18 physical therapy benefits physical therapy benefits
## 84 mental health services benefits ERfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.8888888888888888print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[0 0 0 0 0 0 0 1 0]
## [0 3 0 0 0 0 0 0 0]
## [0 0 3 0 0 0 0 0 0]
## [0 0 0 1 0 1 0 0 0]
## [0 0 0 0 3 0 0 0 0]
## [0 0 0 0 0 2 0 0 0]
## [0 0 0 0 0 0 1 0 0]
## [0 0 0 0 0 0 0 1 0]
## [0 0 0 0 0 0 0 0 2]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## ER 0.00 0.00 0.00 1
## Not Professional 1.00 1.00 1.00 3
## chiropractic benefits 1.00 1.00 1.00 3
## cold stone benefits 1.00 0.50 0.67 2
## cupping benefits 1.00 1.00 1.00 3
## massage benefits 0.67 1.00 0.80 2
## massage gun benefits 1.00 1.00 1.00 1
## mental health services benefits 0.50 1.00 0.67 1
## physical therapy benefits 1.00 1.00 1.00 2
##
## accuracy 0.89 18
## macro avg 0.80 0.83 0.79 18
## weighted avg 0.88 0.89 0.87 18
##
##
## C:\Users\m\Anaconda2\envs\python36\lib\site-packages\sklearn\metrics\classification.py:1437: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.
## 'precision', 'predicted', average, warn_for)Fourth part: Stemmed Tokens & 85/15 Train/Test split & RFC | GBC
Count Vectorizer RFC and GBC
stopwords = nltk.corpus.stopwords.words('english')
ps=nltk.PorterStemmer()
wn=nltk.WordNetLemmatizer()
data=pd.read_csv("benefitsContraindications3.csv", encoding='unicode_escape')
data.columns=['Document','Source','Topic','InternetSearch','Contraindications','RisksSideEffects']data <- read.csv('benefitsContraindications3.csv',sep=',',header=TRUE, na.strings=c('',' ','NA'))
colnames(data)## [1] "Document" "Source" "Topic"
## [4] "InternetSearch" "Contraindications" "risksAdverseEffects"head(data,5)## Document
## 1 Chiropractic adjustments and treatments serve the needs of millions of people around the world.\n\n \n\nAdjustments offer effective, non-invasive and cost-effective solutions to neck and back pain, as well as a myriad of other medical issues.\n\nHave you ever stopped to wonder how many of us suffer from neck and back stiffness or pain?\n\nApart from the obvious discomfort, simple daily tasks such as driving a car, crossing a busy street and picking things up from the floor can become all too challenging for individuals experiencing such pain.\n\nAs anyone who has experienced pain would know, having restricted movement can be debilitating and unfortunately, our busy world doesn?t allow for us to stop.\n\n \nSome of the benefits of long-term chiropractic care include:\n\n Chiropractors can identify mechanical issues that cause spine-related pain and offer a series of adjustments that provide near immediate relief. Following appointments, patients often report feeling their symptoms noticeably better.\n When a chiropractor performs an adjustment, they can help restore movement in joints that have ?locked up?. This becomes possible as treatment allows muscles surrounding joints to relax, thereby reducing joint stiffness.\n Many factors affect health, including exercise patterns, nutrition, sleep, heredity and the environment in which we live. Rather than just treat symptoms of the disease, chiropractic care focuses on a holistic approach to naturally maintain health and resist disease.\n Chiropractic adjustments help restore normal function and movement to the entire body. Many patients report an improvement in their ability to move with efficiency and strength.\n Many patients find delight in the results chiropractic adjustments have on old and chronic injuries. Whether an injury is, in fact, new or old, chiropractic care can help reduce pain, restore mobility and provide quick pain relief to all joints in the body. Such care can help maintain better overall health and thus faster recovery time.\n\nHave you ever noticed that when you are in pain and unable to perform regular or favorite activities, it can put a strain on emotional and mental well-being?\n\nFor example, the increased stress from not being able to properly perform a paid job. This, in turn, can have a negative impact on physical health with increases in heart rate and blood pressure. The domino effect often continues with sleep becoming disturbed, with resulting lethargy and tiredness during the day. Does anyone really feel up to exercising in this state?\n\nChiropractic care is a natural method of healing the body?s communication system and never relies on the use of pharmaceutical drugs or invasive surgery.\n\nDoctors of Chiropractic work collaboratively with other healthcare professionals. Should your condition require the attention of another healthcare profession, that recommendation or referral will be made.\n\n
## 2 \nUnitedHealthcare Combats Opioid Crisis with Non-Opioid Benefits\nPhysical therapy and chiropractic care can prevent or reduce expensive, invasive spinal procedures, such as imaging or surgery, to reduce opioid use and cut costs.\nUnitedHealthcare, opioid, physical therapy, healthcare spending\n\nSource: Thinkstock\nShare on Twitter\n\nBy Kelsey Waddill\n\nOctober 29, 2019 - UnitedHealthcare (UHC) is combatting the opioid epidemic and high healthcare costs with new physical therapy and chiropractic care benefits to prevent, delay, or in some cases substitute for invasive spinal procedures.\n\n?With millions of Americans experiencing low back pain currently or at some point during their lifetimes, we believe this benefit design will help make a meaningful difference by improving health outcomes while reducing costs,? said Anne Docimo, MD, UnitedHealthcare chief medical officer.\n\nLower back pain is in part responsible for sustaining the opioid epidemic and also increases healthcare costs.\nDig Deeper\n\n How Major Payers Provide Substance Abuse Care for Opioid Misuse\n Opioid Overdoses Fall by 2% from 2017 to 2018, CDC Reports\n Physical Therapy, Chiropractic Back Care Cut Opioid Use, Costs\n\nAlthough opioid overdoses fell by two percent from 2017 to 2018 and a legal battles aim to hold pharmaceutical companies accountable, there is no end in sight for the opioid epidemic. Industry professionals are still grappling with the balance between cutting opioid prescriptions will working to reduce patient pain.\n\nCommon conditions such as low back pain bolster the epidemic?s presence, with clinicians still prescribing the opioids against best practice recommendations. According to a recent OptumLabs study, 9 percent of patients with newly diagnosed low back pain are prescribed opioids and lower back pain currently contributes 52 percent to the overall opioid prescription rate.\n\nIn addition to boosting opioids distribution, alternative, invasive lower back pain treatments can significantly impact healthcare spending.\n\nIt is not new information that physical therapy and chiropractic care are effective, lower cost alternatives to spinal imaging or surgery. However, payers are still in the process of adopting the method.\n\nTo counteract the high-cost, high-risk potential of using opioids to treat back pain, UHC created a benefit that does not rely on medication or technology but rather on physical therapy and chiropractic care.\n\nThe benefit allows eligible employers to offer physical therapist and chiropractor visits with no out-of-pocket costs. Members who already receive physical therapist and chiropractic care benefits under UHC?s employer-sponsored health plans and who have maxed out their visits will not receive additional visits under this benefit.\n\nHowever, for those who still have visits to use and who choose physical therapy or chiropractic care over other forms of treatment, the copay or deductible for those visits will be waived and they will receive three visits at no cost.\n\nUHC has high expectations for the fiscal and physical impacts of this benefit.\n\nAccording to UHC?s analysis, the health payer expects that by 2021, opioid use will decrease by 19 percent. Spinal imaging test frequency and spinal surgeries will be reduced by 22 percent and 21 percent, respectively. In addition to these specific goals, UHC hopes to see a decrease in the overall cost of spinal care.\n\nThe same OptumLabs study demonstrated that UHC?s expectations are not without precedent.\n\nThe study looked at the correlation between out-of-pocket costs and patient utilization of noninvasive treatments. Researchers discovered that members whose copay was over $30 were a little under 30 percent less likely to choose physical therapy as opposed to more invasive treatments.\n\nAn American Journal of Managed Care study in June 2019 found that patients with high deductibles, typically over $1,000, were less likely to visit physical therapy.\n\nEligible employers may be brand new or renewing their membership. They must be fully insured and over 51 or more employees strong. The benefit is currently available in Connecticut, Florida, Georgia, New York, and North Carolina.\n\nHowever, UHC plans to expand the benefit from 2020 into 2021. By the end of this expansion period, the benefit will also be available to self-funded employers and organizations with an employee population between 2 and 50. The benefit will span ten states, primarily in the southeast.\n\n?This new benefit design may help encourage people with low back pain to get the right care at the right time and in the right setting, helping expand access to evidence-based and more affordable treatments,? said Docimo.
## 3 The Safety of Chiropractic Adjustments\nBy Lana Barhum\nMedically reviewed by Richard N. Fogoros, MD\nUpdated on January 31, 2020\nOrthopedics\nMore in Orthopedics\n\n Home Office Ergonomics\n Sprains & Strains\n Fractures & Broken Bones\n Physical Therapy\n Orthopedic Surgery\n Osteoporosis\n Pediatric Orthopedics\n Sports Injuries\n\nView All\nIn This Article\n\n Chiropractic Adjustment\n What Research Shows\n Safety\n\nChiropractic adjustment, also called spinal manipulation, is a procedure done by a chiropractor using the hands or small instruments to apply controlled force to a spinal joint. The goal is to improve spinal motion and physical function of the entire body. Chiropractic adjustment is safe when performed by someone who is properly trained and licensed to practice chiropractic care. Complications are rare, but they are possible. Learn more about both the benefits and risks.\nChiropractic adjustment\nVerywell / Brianna Gilmartin \nChiropractic Adjustment\n\nOne of the most important reasons people seek chiropractic care is because it is a completely drug-free therapy. Someone dealing with joint pain, back pain, or headaches might consider visiting a chiropractor.\n\nThe goal of chiropractic adjustment is to place the body into a proper position so the body can heal itself. Treatments are believed to reduce stress on the immune system, reducing the potential for disease. Chiropractic care aims to address the entire body, including a person?s ability to move, perform, and even think.\nWhat Research Shows\n\nMany people wonder how helpful chiropractic care is in treating years of trauma and poor posture. There have been numerous studies showing the therapeutic benefits of chiropractic care.\nSciatica\n\nSciatica is a type of pain affecting the sciatic nerve, the large nerve extending from the low back down the back of the legs. Other natural therapies don?t always offer relief and most people want to avoid steroid injections and surgery, so they turn to chiropractic care.\n\nA double-blind trial reported in the Spine Journal compared active and simulated chiropractic manipulations in people with sciatic nerve pain. Active manipulations involved the patient laying down and receiving treatment from a chiropractor. Stimulated manipulations involved electrical muscle stimulation with electrodes placed on the skin to send electrical pulses to different parts of the body.\n\nThe researchers determined active manipulation offered more benefits than stimulated. The people who received active manipulations experienced fewer days of moderate or severe pain and other sciatica symptoms. They also reported no adverse effects.\nNeck Pain\n\nOne study reported in the Annals of Internal Medicine looked at different therapies for treating neck pain. They divided 272 study participants into three groups: one that received spinal manipulation from a chiropractic doctor, a second group given over-the-counter (OTC) pain relievers, narcotics, and muscle relaxers, and a third group who did at-home exercises. \n\nAfter 12 weeks, patients reported a 75% pain reduction, with the chiropractic treatment group achieving the most improvement. About 57% of the chiropractic group achieved pain reduction, while 48% received pain reduction from exercising, and 33% from medication.\n\nAfter one year, 53% of the drug-free groups continued to report pain relief compared to only 38% of those taking pain medications. \nHeadaches\n\nCervicogenic headaches and migraines are commonly treated by chiropractors. Cervicogenic headaches are often called secondary headaches because pain is usually referred from another source, usually the neck. Migraine headaches cause severe, throbbing pain and are generally experienced on one side of the head. There are few non-medicinal options for managing both types of chronic headaches.\n\nResearch reported in the Journal of Manipulative and Physiological Therapeutics suggests chiropractic care, specifically spinal manipulation, can improve migraines and cervicogenic headaches. \nFrozen Shoulder\n\nFrozen shoulder affects the shoulder joint and involves pain and stiffness that develops gradually and gets worse. Frozen shoulder can be quite painful, and treatment involves preserving as much range of motion in the shoulder as possible and managing pain.\n\nA clinical trial reported in the Journal of Chiropractic Medicine described how patients suffering from frozen shoulder responded to chiropractic treatment. Of the 50 patients, 16 completely recovered, 25 showed a 75 to 90% improvement, and eight showed a 50 to 75% improvement. Only one person showed zero to 50% improvement. The researchers concluded most people can get improvement by treating frozen shoulder with chiropractic treatment.\nPreventing Need for Surgery\n\nChiropractic care may reduce the need for back surgery. Guidelines reported in the Journal of the American Medical Association suggest that it's reasonable for people suffering from back pain to try spinal manipulation before deciding on surgical intervention.\nLow Back Pain\n\nStudies have shown chiropractic care, including spinal manipulation, can provide relief from mild to moderate low back pain. In fact, spinal manipulation may work as well as other standard treatments, including pain-relief medications.\n\nA 2011 review of 26 clinical trials looked at the effectiveness of different treatments for chronic low back pain. What they found was that spinal manipulation is just as effective as other treatments for reducing back pain and improving function.\nSafety\n\n\nWhen chiropractors are correctly trained and licensed, chiropractic care is safe. Mild side effects are to be expected and include temporary soreness, stiffness, and tenderness in the treated area. However, you still want to do your research. Ask for a referral from your doctor. Look at the chiropractor?s website, including patient reviews. Meet with the chiropractor to discuss his or her treatment practices and ask about possible adverse effects related to treatment.\n\nIf you decide a chiropractor isn?t for you, consider seeing an osteopathic doctor. Osteopaths are fully licensed doctors who can practice all areas of medicine. They have received special training on the musculoskeletal system, which includes manual readjustments, myofascial release and other physical manipulation of bones and muscle tissues.
## 4 Advanced Chiropractic Relief: 8 Key Benefits of Chiropractor Care\n\nAre you one of the 50 million Americans who suffer from chronic pain? If so you?re probably intimately familiar with the feeling of pure desperation that can arise from an inability to find relief.\n\nIn addition to physical issues, chronic pain can cause anxiety, depression, and more. However, there could be a light at the end of the tunnel. Many people are finding advanced chiropractic relief that is completely changing their lives.\n\nYour body is a world in itself. At this very moment, more than a million chemical reactions are taking place in your body. It manufactures energy, it regulates your heartbeat, your breathing and it regenerates and heals itself. Everything takes place without your conscious knowledge, without you controlling it voluntarily. The master system that controls it all is your nervous system.\n\nThe nervous system is made out of your brain, spinal cord and all your nerves.\n\nThe energy that flows through your nervous system in your body is like electricity. In order to have that electric flow normally and freely, we need to have a well functioning spine. Whenever you have disruption of that flow, disease happens. That would be the case when your spine is misaligned or is not moving properly.\n\nDid you know that 90% of stimulation and nutrition to the brain is generated by the movement of the spine?\n\nThe more mechanically distorted a person is, the less energy is available for thinking, metabolism and healing.\n\nThis is why it is so important to have a healthy spine, a proper posture, to exercise, to eat properly ? all of it truly matters for your quality of life.\nChiropractors localize the areas of your spine that do not move properly ? referred to as vertebral subluxations ? and adjust them with a specific high speed, but yet gentle, thrust to improve spinal motion.\n\nWant to learn about some of the ways chiropractic care can help you? Keep reading for insight into some of the key benefits of seeing a chiropractor.\n\nThe benefits of chiropractic care are numerous:\n\n1. Lower Blood Pressure\n\nStudies show that chiropractic treatment can lower your blood pressure. Sometimes, this works just as well as a prescription blood pressure medication! This benefit can also last for as long as six months after treatment.\n\nHigh blood pressure can cause an array of serious side effects like nausea, fatigue, dizziness, and anxiety. Sufferers who haven?t found relief should consider consulting with a chiropractor. A chiropractic adjustment may be the solution.\n\nSome studies have shown that chiropractic adjustments can also help patients who are suffering from low blood pressure.\n\n2. Reduced Inflammation\n\nIn many cases, joint issues, pain, and tension are caused by inflammation in the body. Chiropractic adjustments can reduce inflammation.\n\nThis leads to relief of muscle tension, chronic back pain, and joint pain. These adjustments can sometimes also slow the progression of inflammation-related diseases, like arthritis.\n\n3. Better Sleep\n\nPatients who receive chiropractic adjustments report a significant improvement in their sleep patterns. If you regularly suffer from insomnia, visiting a chiropractor regularly may help. Also, when you experience pain relief, this will help you get a restful night?s sleep.\n\n4. Digestive Relief\n\nChiropractors often give nutritional advice as part of their services. However, this isn?t the only way that they provide patients with digestive relief.\n\nAdjusting the thoraco-lumbar spine restores the neurological function of your digestive system. Regular adjustments can help with chronic digestive issues.\n\n5. Stress Release\n\nEveryday life can cause muscle cramping, inflammation, and more. When you?re sore from working at a computer, heavy lifting, or just dealing with emotional stress, a chiropractic adjustment can help. This leads to greater comfort and advanced pain relief.\n\n6. Improvement of Neurological Conditions\n\nA chiropractic adjustment can also increase blood flow to the brain and increase the flow of cerebral spinal fluid. This means that patients suffering from neurological conditions like epilepsy and multiple sclerosis can significantly benefit from regular adjustments.\n\nThis is a relatively new area of study, but the potential is huge. Those suffering from these conditions will want to do some research. It?s important to find the best chiropractor in their area with experience dealing with these specific types of cases.\n\n7. Chiropractic care can improve communication from your brain to your muscles\n\nResearch seems to show that chiropractic care can improve your brain-body communication, helping your brain to be more aware of what is going on in the body so it can control your body better.\n\nBetter health, more energy and vitality are some of the positive effects of getting your spine adjusted. It sets your vertebrae back into motion freeing up the energy that travels through your nerves.\n\nChiropractic care is a partnership. The results patients want is a combination of what the chiropractor does and what the patient does.\n\nThere are many good things that can be changed and improved for a better lifestyle: exercise, good nutrition, good mental attitude and spinal adjustments.\n\nYour whole body will work better by having your nervous system free of interference. That is the essence of chiropractic care and is designed for you and your family.\n\n8. Pain Relief\n\nPerhaps the most well-known benefit of going to a chiropractor is pain relief. Adjustments can help with a huge array of painful conditions including the following.\n\nNeck and Lower Back Pain\n\nAdjustments are the most effective non-invasive pain relief method for this type of pain. They may help patients avoid having to take prescription pain management drugs.\n\nSciatica\n\nTreatments help relieve pressure on the nerve. This results in less severe pain that lasts for a fewer number of days.\n\nHeadaches\n\nChiropractic adjustments help headaches and migraines. They do this by treating back misalignment, muscle tension, and stress. Cervical spine manipulation was associated with significant improvement in headache outcomes in trials involving patients with neck pain and/or neck dysfunction and headache.\n\nChronic headaches can result from the abnormal positioning of the head and can be worsened from neck pressure and movement. Chiropractic removes the interference whether it may be from the distant muscle tightness in the back causing strain on your spine or an abnormal lordotic cervical curve and moving vertebrae.\nChiropractic care can reduce the duration of headaches, lower their intensity when they do occur and limit the frequency of their occurrence all together.\n\nMenstrual cramps\n\nChiropractic treatment removes tension from the pelvis and sacrum. It also regulates the neurological function communicating with the reproductive organs. Adjustments can also relieve the bloating, cramping, and pain associated with menstrual cramps\n\nAnyone who has tried traditional medical treatments and has been unable to find pain relief should experiment with chiropractic care. More often than not, you?ll be pleasantly surprised!\n\nBonus: Advanced Chiropractic Relief\n\nIn addition to the benefits listed above, adjustments can bring advanced chiropractic relief for a wide variety of other conditions as well as overall life improvement. A few examples include:\n\nScoliosis ? adjustments have shown to help with the pain, reduced range of motion, abnormal posture, and even difficulty breathing caused by this abnormal curvature of the spine\n\nVertigo ? an adjustment can help realign and balance the spine, thereby reducing the dizziness, nausea, and disorientation caused by vertigo\n\nSinus and allergy relief ? adjusting the upper cervical spine can help drain the sinuses and provide immediate and lasting relief from both long-term and seasonal allergies\n\nExpectant mothers ? women can experience relief from pain and morning sickness and are better able to maintain proper posture during and after pregnancy\n\nChildren?s issues ? treatments have been shown to help children with acid reflux, cholic, and ear infections\nAthletic performance ? the reduction in pain and inflammation is particularly beneficial for professional and amateur athletes\n\nStimulates the immune system ? chiropractic care helps to boost the immune system, speeding up the healing process following illnesses or injuries. One of the most important studies showing the positive effect chiropractic care can have on the immune system and general health was performed by Ronald Pero, Ph.D., chief of cancer prevention research at New York?s Preventive Medicine Institute and professor of medicine at New York University. Dr. Pero measured the immune systems of people under chiropractic care as compared to those in the general population and those with cancer and other serious diseases.\n\nIn his initial three-year study of 107 individuals who had been under chiropractic care for five years or more, the chiropractic patients were found to have a 200% greater immune competence than people who had not received chiropractic care, and 400% greater immune competence than people with cancer and other serious diseases. The immune system superiority of those under chiropractic care did not diminish with age.\n\nDr. Pero stated: ?When applied in a clinical framework, I have never seen a group other than this chiropractic group to experience a 200% increase over the normal patients. This is why it is so dramatically important. We have never seen such a positive improvement in a group.?\n\nAs you can see, there are almost limitless benefits to seeking chiropractic treatment. If you haven?t tried it yet, what are you waiting for?\n\nThere?s no need to accept pain and discomfort as a normal part of life. You have nothing to lose and everything to gain, so it only makes sense to find out more about this possibly life-changing approach to improving your health and wellness.
## 5 Heading to the spa can be a pampering treat, but it can also be a huge boost to your health and wellness! Massage therapy can relieve all sorts of ailments ? from physical pain, to stress and anxiety. People who choose to supplement their healthcare regimen with regular massages will not only enjoy a relaxing hour or two at the spa, but they will see the benefits carry through the days and weeks after the appointment!\n\n1\n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\nThese are the 10 most common benefits reported from massage therapy:\n\n1. Reduce Stress\n\nA relaxing day at the spa is a great way to unwind and de-stress. However, clients are sure to notice themselves feeling relaxed and at ease for days and even weeks after their appointments!\n\n \n\n2. Improve Circulation\n\nLoosening muscles and tendons allows increased blood flow throughout the body. Improving your circulation can have a number of positive effects on the rest of your body, including reduced fatigue and pain management!\n\n \n\n3. Reduce Pain\n\nMassage therapy is great for working out problem areas like lower back pain and chronic stiffness. A professional therapist will be able to accurately target the source of your pain and help achieve the perfect massage regimen.\n\n \n\n3\n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n4. Eliminate Toxins\n\nStimulating the soft tissues of your body will help to release toxins through your blood and lymphatic systems.\n\n \n\n5. Improve Flexibility\n\nMassage therapy will loosen and relax your muscles, helping your body to achieve its full range of movement potential.\n\n \n\n6. Improve Sleep\n\nA massage will encourage relaxation and boost your mood. Going to bed with relaxed and loosened muscles promotes more restful sleep, and you?ll feel less tired in the morning!\n\n \n\n7. Enhance Immunity\n\nStimulation of the lymph nodes re-charges the body?s natural defense system.\n\n \n\n8. Reduce Fatigue\n\nMassage therapy is known to boost mood and promote better quality sleep, thus making you feel more rested and less worn-out at the end of the day.\n\n \n\n2\n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n \n\n9. Alleviate Depression and Anxiety\n\nMassage therapy can help to release endorphins in your body, helping you to feel happy, energized, and at ease.\n\n \n\n10. Reduce post-surgery and post-injury swelling\n\nA professional massage is a great way to safely deal with a sports injury or post-surgery rehabilitation.\n\nDo you think that massage therapy could help you find relief in any of these areas? What improvements would you like to see in your health? Contact us today with your questions about massage therapy and see how we can help you get on the path to improved health and wellness!
## Source
## 1 https://coremedicalohio.com/benefits-of-long-term-chiropractic-care/?utm_source=ReviveOldPost&utm_medium=social&utm_campaign=ReviveOldPost
## 2 https://healthpayerintelligence.com/news/unitedhealthcare-combats-opioid-crisis-with-non-opioid-benefits
## 3 https://www.verywellhealth.com/is-chiropractic-adjustment-safe-4588279
## 4 https://hafkeychiropractic.com/advanced-chiropractic-relief-8-key-benefits-of-chiropractor-care/
## 5 https://www.urbannirvana.com/10-benefits-massage-therapy/
## Topic InternetSearch Contraindications
## 1 chiropractic benefits <NA> <NA>
## 2 chiropractic benefits <NA> <NA>
## 3 chiropractic benefits <NA> <NA>
## 4 chiropractic benefits <NA> <NA>
## 5 massage benefits google <NA>
## risksAdverseEffects
## 1 <NA>
## 2 <NA>
## 3 Risks and side effects associated with chiropractic adjustments may include:\n\n temporary headaches\n fatigue after treatment\n discomfort in parts of the body that were treated\n\nRare but serious risks associated with chiropractic adjustment include:\n\n stroke\n cauda equina syndrome, a condition involving pinched nerves in the lower part of the spinal canal\n worsening of herniated disks (although research isn't conclusive)\n\nIn addition to effectiveness, research has focused on the safety of chiropractic treatments, mainly spinal manipulation. \n\nOne 2017 review of 250 articles looked at serious adverse events and benign events associated with chiropractic care. Based on the evidence the researchers reviewed, serious adverse events accounted for one out of every two million spinal manipulations to 13 per 10,00 patients. Serious adverse events included spinal or neurological problems and cervical arterial strokes (dissection of any of the arteries in the neck).\n\nBenign events were more common and included more pain and higher levels of neck problems, but most were short-term problems.\n\nThe researchers confirmed serious adverse events were rare and often related to other preexisting conditions, while benign events are more common. However, the reasons for any types of adverse events are unknown.\n\nA second 2017 review looked 118 articles and found frequently described adverse events include stroke, headache and vertebral artery dissection (cervical arterial stroke). Forty-six percent of the reviews determined that spinal manipulation was safe, while 13% expressed concern of harm. The remaining studies were unclear or neutral. While the researchers did not offer an overall conclusion, they determined spinal manipulation can significantly be helpful, and some risk does exist.\nA Word From Verywell
## 4 <NA>
## 5 <NA>def count_punct(text):
count=sum([1 for char in text if char in string.punctuation])
return round(count/(len(text)-text.count(" ")),3)*100data['body_length']=data['Document'].apply(lambda x: len(x)-x.count(" "))
data['punct%']= data['Document'].apply(lambda x: count_punct(x))
def clean_text(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=[ps.stem(word) for word in tokens if word not in stopwords]#these are lists for the count vectorizer
return text
def lemmatize(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=[wn.lemmatize(word) for word in tokens if word not in stopwords]#these are lists for the count vectorizer
return text
data['Cleaned_text']=data['Document'].apply(lambda x: clean_text(x))
data['Lemmatized']=data['Document'].apply(lambda x: lemmatize(x))
data.head()## Document ... Lemmatized
## 0 Chiropractic adjustments and treatments serve ... ... [chiropractic, adjustment, treatment, serve, n...
## 1 \r\nUnitedHealthcare Combats Opioid Crisis wit... ... [, unitedhealthcare, combat, opioid, crisis, n...
## 2 The Safety of Chiropractic Adjustments\r\nBy ... ... [, safety, chiropractic, adjustment, lana, bar...
## 3 Advanced Chiropractic Relief: 8 Key Benefits o... ... [advanced, chiropractic, relief, 8, key, benef...
## 4 Heading to the spa can be a pampering treat, b... ... [heading, spa, pampering, treat, also, huge, b...
##
## [5 rows x 10 columns]data.to_csv('dataCleanLemm.csv')
#DATA = pd.read_csv('dataCleanLemm.csv', encoding='unicode_escape')DATA <- read.csv('dataCleanLemm.csv', sep=',', header=TRUE, na.strings=c('',' ','NA'), row.names=1)
colnames(DATA)## [1] "Document" "Source" "Topic"
## [4] "InternetSearch" "Contraindications" "RisksSideEffects"
## [7] "body_length" "punct." "Cleaned_text"
## [10] "Lemmatized"head(DATA,2)## Document
## 0 Chiropractic adjustments and treatments serve the needs of millions of people around the world.\n\n \n\nAdjustments offer effective, non-invasive and cost-effective solutions to neck and back pain, as well as a myriad of other medical issues.\n\nHave you ever stopped to wonder how many of us suffer from neck and back stiffness or pain?\n\nApart from the obvious discomfort, simple daily tasks such as driving a car, crossing a busy street and picking things up from the floor can become all too challenging for individuals experiencing such pain.\n\nAs anyone who has experienced pain would know, having restricted movement can be debilitating and unfortunately, our busy world doesn?t allow for us to stop.\n\n \nSome of the benefits of long-term chiropractic care include:\n\n Chiropractors can identify mechanical issues that cause spine-related pain and offer a series of adjustments that provide near immediate relief. Following appointments, patients often report feeling their symptoms noticeably better.\n When a chiropractor performs an adjustment, they can help restore movement in joints that have ?locked up?. This becomes possible as treatment allows muscles surrounding joints to relax, thereby reducing joint stiffness.\n Many factors affect health, including exercise patterns, nutrition, sleep, heredity and the environment in which we live. Rather than just treat symptoms of the disease, chiropractic care focuses on a holistic approach to naturally maintain health and resist disease.\n Chiropractic adjustments help restore normal function and movement to the entire body. Many patients report an improvement in their ability to move with efficiency and strength.\n Many patients find delight in the results chiropractic adjustments have on old and chronic injuries. Whether an injury is, in fact, new or old, chiropractic care can help reduce pain, restore mobility and provide quick pain relief to all joints in the body. Such care can help maintain better overall health and thus faster recovery time.\n\nHave you ever noticed that when you are in pain and unable to perform regular or favorite activities, it can put a strain on emotional and mental well-being?\n\nFor example, the increased stress from not being able to properly perform a paid job. This, in turn, can have a negative impact on physical health with increases in heart rate and blood pressure. The domino effect often continues with sleep becoming disturbed, with resulting lethargy and tiredness during the day. Does anyone really feel up to exercising in this state?\n\nChiropractic care is a natural method of healing the body?s communication system and never relies on the use of pharmaceutical drugs or invasive surgery.\n\nDoctors of Chiropractic work collaboratively with other healthcare professionals. Should your condition require the attention of another healthcare profession, that recommendation or referral will be made.\n\n
## 1 \nUnitedHealthcare Combats Opioid Crisis with Non-Opioid Benefits\nPhysical therapy and chiropractic care can prevent or reduce expensive, invasive spinal procedures, such as imaging or surgery, to reduce opioid use and cut costs.\nUnitedHealthcare, opioid, physical therapy, healthcare spending\n\nSource: Thinkstock\nShare on Twitter\n\nBy Kelsey Waddill\n\nOctober 29, 2019 - UnitedHealthcare (UHC) is combatting the opioid epidemic and high healthcare costs with new physical therapy and chiropractic care benefits to prevent, delay, or in some cases substitute for invasive spinal procedures.\n\n?With millions of Americans experiencing low back pain currently or at some point during their lifetimes, we believe this benefit design will help make a meaningful difference by improving health outcomes while reducing costs,? said Anne Docimo, MD, UnitedHealthcare chief medical officer.\n\nLower back pain is in part responsible for sustaining the opioid epidemic and also increases healthcare costs.\nDig Deeper\n\n How Major Payers Provide Substance Abuse Care for Opioid Misuse\n Opioid Overdoses Fall by 2% from 2017 to 2018, CDC Reports\n Physical Therapy, Chiropractic Back Care Cut Opioid Use, Costs\n\nAlthough opioid overdoses fell by two percent from 2017 to 2018 and a legal battles aim to hold pharmaceutical companies accountable, there is no end in sight for the opioid epidemic. Industry professionals are still grappling with the balance between cutting opioid prescriptions will working to reduce patient pain.\n\nCommon conditions such as low back pain bolster the epidemic?s presence, with clinicians still prescribing the opioids against best practice recommendations. According to a recent OptumLabs study, 9 percent of patients with newly diagnosed low back pain are prescribed opioids and lower back pain currently contributes 52 percent to the overall opioid prescription rate.\n\nIn addition to boosting opioids distribution, alternative, invasive lower back pain treatments can significantly impact healthcare spending.\n\nIt is not new information that physical therapy and chiropractic care are effective, lower cost alternatives to spinal imaging or surgery. However, payers are still in the process of adopting the method.\n\nTo counteract the high-cost, high-risk potential of using opioids to treat back pain, UHC created a benefit that does not rely on medication or technology but rather on physical therapy and chiropractic care.\n\nThe benefit allows eligible employers to offer physical therapist and chiropractor visits with no out-of-pocket costs. Members who already receive physical therapist and chiropractic care benefits under UHC?s employer-sponsored health plans and who have maxed out their visits will not receive additional visits under this benefit.\n\nHowever, for those who still have visits to use and who choose physical therapy or chiropractic care over other forms of treatment, the copay or deductible for those visits will be waived and they will receive three visits at no cost.\n\nUHC has high expectations for the fiscal and physical impacts of this benefit.\n\nAccording to UHC?s analysis, the health payer expects that by 2021, opioid use will decrease by 19 percent. Spinal imaging test frequency and spinal surgeries will be reduced by 22 percent and 21 percent, respectively. In addition to these specific goals, UHC hopes to see a decrease in the overall cost of spinal care.\n\nThe same OptumLabs study demonstrated that UHC?s expectations are not without precedent.\n\nThe study looked at the correlation between out-of-pocket costs and patient utilization of noninvasive treatments. Researchers discovered that members whose copay was over $30 were a little under 30 percent less likely to choose physical therapy as opposed to more invasive treatments.\n\nAn American Journal of Managed Care study in June 2019 found that patients with high deductibles, typically over $1,000, were less likely to visit physical therapy.\n\nEligible employers may be brand new or renewing their membership. They must be fully insured and over 51 or more employees strong. The benefit is currently available in Connecticut, Florida, Georgia, New York, and North Carolina.\n\nHowever, UHC plans to expand the benefit from 2020 into 2021. By the end of this expansion period, the benefit will also be available to self-funded employers and organizations with an employee population between 2 and 50. The benefit will span ten states, primarily in the southeast.\n\n?This new benefit design may help encourage people with low back pain to get the right care at the right time and in the right setting, helping expand access to evidence-based and more affordable treatments,? said Docimo.
## Source
## 0 https://coremedicalohio.com/benefits-of-long-term-chiropractic-care/?utm_source=ReviveOldPost&utm_medium=social&utm_campaign=ReviveOldPost
## 1 https://healthpayerintelligence.com/news/unitedhealthcare-combats-opioid-crisis-with-non-opioid-benefits
## Topic InternetSearch Contraindications RisksSideEffects
## 0 chiropractic benefits <NA> <NA> <NA>
## 1 chiropractic benefits <NA> <NA> <NA>
## body_length punct.
## 0 2497 2.3
## 1 4055 2.4
## Cleaned_text
## 0 ['chiropract', 'adjust', 'treatment', 'serv', 'need', 'million', 'peopl', 'around', 'world', 'adjust', 'offer', 'effect', 'noninvas', 'costeffect', 'solut', 'neck', 'back', 'pain', 'well', 'myriad', 'medic', 'issu', 'ever', 'stop', 'wonder', 'mani', 'us', 'suffer', 'neck', 'back', 'stiff', 'pain', 'apart', 'obviou', 'discomfort', 'simpl', 'daili', 'task', 'drive', 'car', 'cross', 'busi', 'street', 'pick', 'thing', 'floor', 'becom', 'challeng', 'individu', 'experienc', 'pain', 'anyon', 'experienc', 'pain', 'would', 'know', 'restrict', 'movement', 'debilit', 'unfortun', 'busi', 'world', 'doesnt', 'allow', 'us', 'stop', 'benefit', 'longterm', 'chiropract', 'care', 'includ', 'chiropractor', 'identifi', 'mechan', 'issu', 'caus', 'spinerel', 'pain', 'offer', 'seri', 'adjust', 'provid', 'near', 'immedi', 'relief', 'follow', 'appoint', 'patient', 'often', 'report', 'feel', 'symptom', 'notic', 'better', 'chiropractor', 'perform', 'adjust', 'help', 'restor', 'movement', 'joint', 'lock', 'becom', 'possibl', 'treatment', 'allow', 'muscl', 'surround', 'joint', 'relax', 'therebi', 'reduc', 'joint', 'stiff', 'mani', 'factor', 'affect', 'health', 'includ', 'exercis', 'pattern', 'nutrit', 'sleep', 'hered', 'environ', 'live', 'rather', 'treat', 'symptom', 'diseas', 'chiropract', 'care', 'focus', 'holist', 'approach', 'natur', 'maintain', 'health', 'resist', 'diseas', 'chiropract', 'adjust', 'help', 'restor', 'normal', 'function', 'movement', 'entir', 'bodi', 'mani', 'patient', 'report', 'improv', 'abil', 'move', 'effici', 'strength', 'mani', 'patient', 'find', 'delight', 'result', 'chiropract', 'adjust', 'old', 'chronic', 'injuri', 'whether', 'injuri', 'fact', 'new', 'old', 'chiropract', 'care', 'help', 'reduc', 'pain', 'restor', 'mobil', 'provid', 'quick', 'pain', 'relief', 'joint', 'bodi', 'care', 'help', 'maintain', 'better', 'overal', 'health', 'thu', 'faster', 'recoveri', 'time', 'ever', 'notic', 'pain', 'unabl', 'perform', 'regular', 'favorit', 'activ', 'put', 'strain', 'emot', 'mental', 'wellb', 'exampl', 'increas', 'stress', 'abl', 'properli', 'perform', 'paid', 'job', 'turn', 'neg', 'impact', 'physic', 'health', 'increas', 'heart', 'rate', 'blood', 'pressur', 'domino', 'effect', 'often', 'continu', 'sleep', 'becom', 'disturb', 'result', 'lethargi', 'tired', 'day', 'anyon', 'realli', 'feel', 'exercis', 'state', 'chiropract', 'care', 'natur', 'method', 'heal', 'bodi', 'commun', 'system', 'never', 'reli', 'use', 'pharmaceut', 'drug', 'invas', 'surgeri', 'doctor', 'chiropract', 'work', 'collabor', 'healthcar', 'profession', 'condit', 'requir', 'attent', 'anoth', 'healthcar', 'profess', 'recommend', 'referr', 'made', '']
## 1 ['', 'unitedhealthcar', 'combat', 'opioid', 'crisi', 'nonopioid', 'benefit', 'physic', 'therapi', 'chiropract', 'care', 'prevent', 'reduc', 'expens', 'invas', 'spinal', 'procedur', 'imag', 'surgeri', 'reduc', 'opioid', 'use', 'cut', 'cost', 'unitedhealthcar', 'opioid', 'physic', 'therapi', 'healthcar', 'spend', 'sourc', 'thinkstock', 'share', 'twitter', 'kelsey', 'waddil', 'octob', '29', '2019', 'unitedhealthcar', 'uhc', 'combat', 'opioid', 'epidem', 'high', 'healthcar', 'cost', 'new', 'physic', 'therapi', 'chiropract', 'care', 'benefit', 'prevent', 'delay', 'case', 'substitut', 'invas', 'spinal', 'procedur', 'million', 'american', 'experienc', 'low', 'back', 'pain', 'current', 'point', 'lifetim', 'believ', 'benefit', 'design', 'help', 'make', 'meaning', 'differ', 'improv', 'health', 'outcom', 'reduc', 'cost', 'said', 'ann', 'docimo', 'md', 'unitedhealthcar', 'chief', 'medic', 'offic', 'lower', 'back', 'pain', 'part', 'respons', 'sustain', 'opioid', 'epidem', 'also', 'increas', 'healthcar', 'cost', 'dig', 'deeper', 'major', 'payer', 'provid', 'substanc', 'abus', 'care', 'opioid', 'misus', 'opioid', 'overdos', 'fall', '2', '2017', '2018', 'cdc', 'report', 'physic', 'therapi', 'chiropract', 'back', 'care', 'cut', 'opioid', 'use', 'cost', 'although', 'opioid', 'overdos', 'fell', 'two', 'percent', '2017', '2018', 'legal', 'battl', 'aim', 'hold', 'pharmaceut', 'compani', 'account', 'end', 'sight', 'opioid', 'epidem', 'industri', 'profession', 'still', 'grappl', 'balanc', 'cut', 'opioid', 'prescript', 'work', 'reduc', 'patient', 'pain', 'common', 'condit', 'low', 'back', 'pain', 'bolster', 'epidem', 'presenc', 'clinician', 'still', 'prescrib', 'opioid', 'best', 'practic', 'recommend', 'accord', 'recent', 'optumlab', 'studi', '9', 'percent', 'patient', 'newli', 'diagnos', 'low', 'back', 'pain', 'prescrib', 'opioid', 'lower', 'back', 'pain', 'current', 'contribut', '52', 'percent', 'overal', 'opioid', 'prescript', 'rate', 'addit', 'boost', 'opioid', 'distribut', 'altern', 'invas', 'lower', 'back', 'pain', 'treatment', 'significantli', 'impact', 'healthcar', 'spend', 'new', 'inform', 'physic', 'therapi', 'chiropract', 'care', 'effect', 'lower', 'cost', 'altern', 'spinal', 'imag', 'surgeri', 'howev', 'payer', 'still', 'process', 'adopt', 'method', 'counteract', 'highcost', 'highrisk', 'potenti', 'use', 'opioid', 'treat', 'back', 'pain', 'uhc', 'creat', 'benefit', 'reli', 'medic', 'technolog', 'rather', 'physic', 'therapi', 'chiropract', 'care', 'benefit', 'allow', 'elig', 'employ', 'offer', 'physic', 'therapist', 'chiropractor', 'visit', 'outofpocket', 'cost', 'member', 'alreadi', 'receiv', 'physic', 'therapist', 'chiropract', 'care', 'benefit', 'uhc', 'employersponsor', 'health', 'plan', 'max', 'visit', 'receiv', 'addit', 'visit', 'benefit', 'howev', 'still', 'visit', 'use', 'choos', 'physic', 'therapi', 'chiropract', 'care', 'form', 'treatment', 'copay', 'deduct', 'visit', 'waiv', 'receiv', 'three', 'visit', 'cost', 'uhc', 'high', 'expect', 'fiscal', 'physic', 'impact', 'benefit', 'accord', 'uhc', 'analysi', 'health', 'payer', 'expect', '2021', 'opioid', 'use', 'decreas', '19', 'percent', 'spinal', 'imag', 'test', 'frequenc', 'spinal', 'surgeri', 'reduc', '22', 'percent', '21', 'percent', 'respect', 'addit', 'specif', 'goal', 'uhc', 'hope', 'see', 'decreas', 'overal', 'cost', 'spinal', 'care', 'optumlab', 'studi', 'demonstr', 'uhc', 'expect', 'without', 'preced', 'studi', 'look', 'correl', 'outofpocket', 'cost', 'patient', 'util', 'noninvas', 'treatment', 'research', 'discov', 'member', 'whose', 'copay', '30', 'littl', '30', 'percent', 'less', 'like', 'choos', 'physic', 'therapi', 'oppos', 'invas', 'treatment', 'american', 'journal', 'manag', 'care', 'studi', 'june', '2019', 'found', 'patient', 'high', 'deduct', 'typic', '1000', 'less', 'like', 'visit', 'physic', 'therapi', 'elig', 'employ', 'may', 'brand', 'new', 'renew', 'membership', 'must', 'fulli', 'insur', '51', 'employe', 'strong', 'benefit', 'current', 'avail', 'connecticut', 'florida', 'georgia', 'new', 'york', 'north', 'carolina', 'howev', 'uhc', 'plan', 'expand', 'benefit', '2020', '2021', 'end', 'expans', 'period', 'benefit', 'also', 'avail', 'selffund', 'employ', 'organ', 'employe', 'popul', '2', '50', 'benefit', 'span', 'ten', 'state', 'primarili', 'southeast', 'new', 'benefit', 'design', 'may', 'help', 'encourag', 'peopl', 'low', 'back', 'pain', 'get', 'right', 'care', 'right', 'time', 'right', 'set', 'help', 'expand', 'access', 'evidencebas', 'afford', 'treatment', 'said', 'docimo']
## Lemmatized
## 0 ['chiropractic', 'adjustment', 'treatment', 'serve', 'need', 'million', 'people', 'around', 'world', 'adjustment', 'offer', 'effective', 'noninvasive', 'costeffective', 'solution', 'neck', 'back', 'pain', 'well', 'myriad', 'medical', 'issue', 'ever', 'stopped', 'wonder', 'many', 'u', 'suffer', 'neck', 'back', 'stiffness', 'pain', 'apart', 'obvious', 'discomfort', 'simple', 'daily', 'task', 'driving', 'car', 'crossing', 'busy', 'street', 'picking', 'thing', 'floor', 'become', 'challenging', 'individual', 'experiencing', 'pain', 'anyone', 'experienced', 'pain', 'would', 'know', 'restricted', 'movement', 'debilitating', 'unfortunately', 'busy', 'world', 'doesnt', 'allow', 'u', 'stop', 'benefit', 'longterm', 'chiropractic', 'care', 'include', 'chiropractor', 'identify', 'mechanical', 'issue', 'cause', 'spinerelated', 'pain', 'offer', 'series', 'adjustment', 'provide', 'near', 'immediate', 'relief', 'following', 'appointment', 'patient', 'often', 'report', 'feeling', 'symptom', 'noticeably', 'better', 'chiropractor', 'performs', 'adjustment', 'help', 'restore', 'movement', 'joint', 'locked', 'becomes', 'possible', 'treatment', 'allows', 'muscle', 'surrounding', 'joint', 'relax', 'thereby', 'reducing', 'joint', 'stiffness', 'many', 'factor', 'affect', 'health', 'including', 'exercise', 'pattern', 'nutrition', 'sleep', 'heredity', 'environment', 'live', 'rather', 'treat', 'symptom', 'disease', 'chiropractic', 'care', 'focus', 'holistic', 'approach', 'naturally', 'maintain', 'health', 'resist', 'disease', 'chiropractic', 'adjustment', 'help', 'restore', 'normal', 'function', 'movement', 'entire', 'body', 'many', 'patient', 'report', 'improvement', 'ability', 'move', 'efficiency', 'strength', 'many', 'patient', 'find', 'delight', 'result', 'chiropractic', 'adjustment', 'old', 'chronic', 'injury', 'whether', 'injury', 'fact', 'new', 'old', 'chiropractic', 'care', 'help', 'reduce', 'pain', 'restore', 'mobility', 'provide', 'quick', 'pain', 'relief', 'joint', 'body', 'care', 'help', 'maintain', 'better', 'overall', 'health', 'thus', 'faster', 'recovery', 'time', 'ever', 'noticed', 'pain', 'unable', 'perform', 'regular', 'favorite', 'activity', 'put', 'strain', 'emotional', 'mental', 'wellbeing', 'example', 'increased', 'stress', 'able', 'properly', 'perform', 'paid', 'job', 'turn', 'negative', 'impact', 'physical', 'health', 'increase', 'heart', 'rate', 'blood', 'pressure', 'domino', 'effect', 'often', 'continues', 'sleep', 'becoming', 'disturbed', 'resulting', 'lethargy', 'tiredness', 'day', 'anyone', 'really', 'feel', 'exercising', 'state', 'chiropractic', 'care', 'natural', 'method', 'healing', 'body', 'communication', 'system', 'never', 'relies', 'use', 'pharmaceutical', 'drug', 'invasive', 'surgery', 'doctor', 'chiropractic', 'work', 'collaboratively', 'healthcare', 'professional', 'condition', 'require', 'attention', 'another', 'healthcare', 'profession', 'recommendation', 'referral', 'made', '']
## 1 ['', 'unitedhealthcare', 'combat', 'opioid', 'crisis', 'nonopioid', 'benefit', 'physical', 'therapy', 'chiropractic', 'care', 'prevent', 'reduce', 'expensive', 'invasive', 'spinal', 'procedure', 'imaging', 'surgery', 'reduce', 'opioid', 'use', 'cut', 'cost', 'unitedhealthcare', 'opioid', 'physical', 'therapy', 'healthcare', 'spending', 'source', 'thinkstock', 'share', 'twitter', 'kelsey', 'waddill', 'october', '29', '2019', 'unitedhealthcare', 'uhc', 'combatting', 'opioid', 'epidemic', 'high', 'healthcare', 'cost', 'new', 'physical', 'therapy', 'chiropractic', 'care', 'benefit', 'prevent', 'delay', 'case', 'substitute', 'invasive', 'spinal', 'procedure', 'million', 'american', 'experiencing', 'low', 'back', 'pain', 'currently', 'point', 'lifetime', 'believe', 'benefit', 'design', 'help', 'make', 'meaningful', 'difference', 'improving', 'health', 'outcome', 'reducing', 'cost', 'said', 'anne', 'docimo', 'md', 'unitedhealthcare', 'chief', 'medical', 'officer', 'lower', 'back', 'pain', 'part', 'responsible', 'sustaining', 'opioid', 'epidemic', 'also', 'increase', 'healthcare', 'cost', 'dig', 'deeper', 'major', 'payer', 'provide', 'substance', 'abuse', 'care', 'opioid', 'misuse', 'opioid', 'overdoses', 'fall', '2', '2017', '2018', 'cdc', 'report', 'physical', 'therapy', 'chiropractic', 'back', 'care', 'cut', 'opioid', 'use', 'cost', 'although', 'opioid', 'overdoses', 'fell', 'two', 'percent', '2017', '2018', 'legal', 'battle', 'aim', 'hold', 'pharmaceutical', 'company', 'accountable', 'end', 'sight', 'opioid', 'epidemic', 'industry', 'professional', 'still', 'grappling', 'balance', 'cutting', 'opioid', 'prescription', 'working', 'reduce', 'patient', 'pain', 'common', 'condition', 'low', 'back', 'pain', 'bolster', 'epidemic', 'presence', 'clinician', 'still', 'prescribing', 'opioids', 'best', 'practice', 'recommendation', 'according', 'recent', 'optumlabs', 'study', '9', 'percent', 'patient', 'newly', 'diagnosed', 'low', 'back', 'pain', 'prescribed', 'opioids', 'lower', 'back', 'pain', 'currently', 'contributes', '52', 'percent', 'overall', 'opioid', 'prescription', 'rate', 'addition', 'boosting', 'opioids', 'distribution', 'alternative', 'invasive', 'lower', 'back', 'pain', 'treatment', 'significantly', 'impact', 'healthcare', 'spending', 'new', 'information', 'physical', 'therapy', 'chiropractic', 'care', 'effective', 'lower', 'cost', 'alternative', 'spinal', 'imaging', 'surgery', 'however', 'payer', 'still', 'process', 'adopting', 'method', 'counteract', 'highcost', 'highrisk', 'potential', 'using', 'opioids', 'treat', 'back', 'pain', 'uhc', 'created', 'benefit', 'rely', 'medication', 'technology', 'rather', 'physical', 'therapy', 'chiropractic', 'care', 'benefit', 'allows', 'eligible', 'employer', 'offer', 'physical', 'therapist', 'chiropractor', 'visit', 'outofpocket', 'cost', 'member', 'already', 'receive', 'physical', 'therapist', 'chiropractic', 'care', 'benefit', 'uhcs', 'employersponsored', 'health', 'plan', 'maxed', 'visit', 'receive', 'additional', 'visit', 'benefit', 'however', 'still', 'visit', 'use', 'choose', 'physical', 'therapy', 'chiropractic', 'care', 'form', 'treatment', 'copay', 'deductible', 'visit', 'waived', 'receive', 'three', 'visit', 'cost', 'uhc', 'high', 'expectation', 'fiscal', 'physical', 'impact', 'benefit', 'according', 'uhcs', 'analysis', 'health', 'payer', 'expects', '2021', 'opioid', 'use', 'decrease', '19', 'percent', 'spinal', 'imaging', 'test', 'frequency', 'spinal', 'surgery', 'reduced', '22', 'percent', '21', 'percent', 'respectively', 'addition', 'specific', 'goal', 'uhc', 'hope', 'see', 'decrease', 'overall', 'cost', 'spinal', 'care', 'optumlabs', 'study', 'demonstrated', 'uhcs', 'expectation', 'without', 'precedent', 'study', 'looked', 'correlation', 'outofpocket', 'cost', 'patient', 'utilization', 'noninvasive', 'treatment', 'researcher', 'discovered', 'member', 'whose', 'copay', '30', 'little', '30', 'percent', 'le', 'likely', 'choose', 'physical', 'therapy', 'opposed', 'invasive', 'treatment', 'american', 'journal', 'managed', 'care', 'study', 'june', '2019', 'found', 'patient', 'high', 'deductible', 'typically', '1000', 'le', 'likely', 'visit', 'physical', 'therapy', 'eligible', 'employer', 'may', 'brand', 'new', 'renewing', 'membership', 'must', 'fully', 'insured', '51', 'employee', 'strong', 'benefit', 'currently', 'available', 'connecticut', 'florida', 'georgia', 'new', 'york', 'north', 'carolina', 'however', 'uhc', 'plan', 'expand', 'benefit', '2020', '2021', 'end', 'expansion', 'period', 'benefit', 'also', 'available', 'selffunded', 'employer', 'organization', 'employee', 'population', '2', '50', 'benefit', 'span', 'ten', 'state', 'primarily', 'southeast', 'new', 'benefit', 'design', 'may', 'help', 'encourage', 'people', 'low', 'back', 'pain', 'get', 'right', 'care', 'right', 'time', 'right', 'setting', 'helping', 'expand', 'access', 'evidencebased', 'affordable', 'treatment', 'said', 'docimo']from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(data[['Document','body_length','punct%','Cleaned_text','Lemmatized']],data['Topic'],test_size=0.15)from sklearn.feature_extraction.text import CountVectorizercount_vect=CountVectorizer(analyzer=clean_text)
count_vect_fit=count_vect.fit(X_train['Document'])
count_train=count_vect_fit.transform(X_train['Document'])
count_test=count_vect_fit.transform(X_test['Document'])len(count_vect_fit.get_feature_names())## 3743count_vect_fit.get_feature_names()[200:350]## ['addthi', 'adelin', 'adequ', 'adhes', 'adjunct', 'adjust', 'administ', 'administr', 'adminmcb', 'admiss', 'admit', 'adolesc', 'adopt', 'ador', 'adult', 'adults1', 'advanc', 'advantag', 'advers', 'advertis', 'advic', 'advis', 'advisori', 'advoc', 'aerob', 'affair', 'affect', 'affili', 'affirm', 'afford', 'afterhour', 'afterward', 'agakian', 'age', 'agenc', 'agephys', 'agerel', 'aggress', 'agil', 'ago', 'agre', 'ahead', 'aid', 'ailment', 'aim', 'air', 'airport', 'aka', 'al', 'alchemist', 'alcohol', 'alert', 'align', 'alik', 'allerg', 'allergi', 'allevi', 'allianc', 'allison', 'allow', 'almost', 'alon', 'along', 'alongsid', 'alreadi', 'alright', 'also', 'alter', 'altern', 'alternatingli', 'although', 'altogeth', 'alway', 'alzheim', 'amateur', 'amaz', 'amazon', 'amazoncom', 'ambul', 'amelior', 'america', 'american', 'ami', 'among', 'amount', 'amplitud', 'amta', 'analysi', 'analyz', 'anatomi', 'ancient', 'andor', 'anecdot', 'anemia', 'anesthet', 'angel', 'anger', 'angl', 'angri', 'anhedonia', 'anim', 'ankl', 'ann', 'annal', 'anni', 'announc', 'annual', 'anosognosia', 'anoth', 'answer', 'antibiot', 'antibodi', 'anticip', 'anticoagul', 'antidepress', 'antiinflammatori', 'antivir', 'anxieti', 'anxietydepress', 'anxietyfre', 'anxiou', 'anyon', 'anyth', 'anywher', 'apa', 'apart', 'appeal', 'appear', 'appendix', 'appetit', 'appli', 'applianc', 'applic', 'appoint', 'approach', 'appropri', 'approv', 'approxim', 'apr', 'april', 'apta', 'arab', 'archiv', 'area', 'arent', 'aris', 'arm', 'armonk', 'aromatherapi', 'around']count_train_vect=pd.concat([X_train[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(count_train.toarray())],axis=1)
count_test_vect=pd.concat([X_test[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(count_test.toarray())],axis=1)count_train_vect.head()## Document body_length ... 3741 3742
## 0 Cupping Therapy\r\nPrivacy & Trust Info\r\nCor... 7684 ... 1 0
## 1 \r\nFive Warning Signs of Mental Illness\r\n\r... 6991 ... 0 0
## 2 This futuristic ?gun? could be more effective ... 5308 ... 0 0
## 3 lucy should experience the mindnumbing pleasur... 49 ... 0 0
## 4 Learn All About Cupping Therapy And The Many ... 2204 ... 0 0
##
## [5 rows x 3748 columns]from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import precision_recall_fscore_support as score
import timerf=RandomForestClassifier(n_estimators=150, max_depth=None, n_jobs=-1)
start=time.time()
rf_model=rf.fit(count_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=rf_model.predict(count_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 3 chiropractic benefits chiropractic benefits
## 78 Not Professional Not Professional
## 2 chiropractic benefits chiropractic benefits
## 35 cupping benefits cupping benefits
## 62 cold stone benefits cold stone benefits
## 51 massage gun benefits massage gun benefits
## 68 Not Professional Not Professional
## 80 Not Professional Not Professional
## 4 massage benefits massage benefits
## 8 Not Professional massage benefits
## 69 Not Professional Not Professional
## 71 Not Professional Not Professional
## 20 physical therapy benefits physical therapy benefits
## 5 massage benefits massage benefitsfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.9285714285714286print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[5 0 0 0 0 0 0]
## [0 2 0 0 0 0 0]
## [0 0 1 0 0 0 0]
## [0 0 0 1 0 0 0]
## [1 0 0 0 2 0 0]
## [0 0 0 0 0 1 0]
## [0 0 0 0 0 0 1]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 0.83 1.00 0.91 5
## chiropractic benefits 1.00 1.00 1.00 2
## cold stone benefits 1.00 1.00 1.00 1
## cupping benefits 1.00 1.00 1.00 1
## massage benefits 1.00 0.67 0.80 3
## massage gun benefits 1.00 1.00 1.00 1
## physical therapy benefits 1.00 1.00 1.00 1
##
## accuracy 0.93 14
## macro avg 0.98 0.95 0.96 14
## weighted avg 0.94 0.93 0.92 14gb=GradientBoostingClassifier(n_estimators=150,max_depth=11)
start=time.time()
gb_model=gb.fit(count_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=gb_model.predict(count_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 3 chiropractic benefits chiropractic benefits
## 78 Not Professional Not Professional
## 2 chiropractic benefits chiropractic benefits
## 35 cupping benefits cupping benefits
## 62 cold stone benefits cold stone benefits
## 51 massage gun benefits massage gun benefits
## 68 Not Professional Not Professional
## 80 Not Professional Not Professional
## 4 massage benefits massage benefits
## 8 Not Professional massage benefits
## 69 Not Professional Not Professional
## 71 Not Professional Not Professional
## 20 physical therapy benefits physical therapy benefits
## 5 massage benefits massage benefitsfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.9285714285714286print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[5 0 0 0 0 0 0]
## [0 2 0 0 0 0 0]
## [0 0 1 0 0 0 0]
## [0 0 0 1 0 0 0]
## [1 0 0 0 2 0 0]
## [0 0 0 0 0 1 0]
## [0 0 0 0 0 0 1]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 0.83 1.00 0.91 5
## chiropractic benefits 1.00 1.00 1.00 2
## cold stone benefits 1.00 1.00 1.00 1
## cupping benefits 1.00 1.00 1.00 1
## massage benefits 1.00 0.67 0.80 3
## massage gun benefits 1.00 1.00 1.00 1
## physical therapy benefits 1.00 1.00 1.00 1
##
## accuracy 0.93 14
## macro avg 0.98 0.95 0.96 14
## weighted avg 0.94 0.93 0.92 14TF-IDF RFC and GBC
stopwords = nltk.corpus.stopwords.words('english')
ps=nltk.PorterStemmer()
wn=nltk.WordNetLemmatizer()
data=pd.read_csv("benefitsContraindications3.csv", encoding='unicode_escape')
data.columns=['Document','Source','Topic','InternetSearch','Contraindications','RisksSideEffects']def count_punct(text):
count=sum([1 for char in text if char in string.punctuation])
return round(count/(len(text)-text.count(" ")),3)*100data['body_length']=data['Document'].apply(lambda x: len(x)-x.count(" "))
data['punct%']= data['Document'].apply(lambda x: count_punct(x))
def clean_text(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=[ps.stem(word) for word in tokens if word not in stopwords]#these are lists for the count vectorizer
return text
def lemmatize(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=[wn.lemmatize(word) for word in tokens if word not in stopwords]#these are lists for the count vectorizer
return text
data['Cleaned_text']=data['Document'].apply(lambda x: clean_text(x))
data['Lemmatized']=data['Document'].apply(lambda x: lemmatize(x))
data.head()## Document ... Lemmatized
## 0 Chiropractic adjustments and treatments serve ... ... [chiropractic, adjustment, treatment, serve, n...
## 1 \r\nUnitedHealthcare Combats Opioid Crisis wit... ... [, unitedhealthcare, combat, opioid, crisis, n...
## 2 The Safety of Chiropractic Adjustments\r\nBy ... ... [, safety, chiropractic, adjustment, lana, bar...
## 3 Advanced Chiropractic Relief: 8 Key Benefits o... ... [advanced, chiropractic, relief, 8, key, benef...
## 4 Heading to the spa can be a pampering treat, b... ... [heading, spa, pampering, treat, also, huge, b...
##
## [5 rows x 10 columns]
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(data[['Document','body_length','punct%','Cleaned_text','Lemmatized']],data['Topic'],test_size=0.15)
from sklearn.feature_extraction.text import TfidfVectorizer
tfidf_vect=TfidfVectorizer(analyzer=clean_text)
tfidf_vect_fit=tfidf_vect.fit(X_train['Document'])
tfidf_train=tfidf_vect_fit.transform(X_train['Document'])
tfidf_test=tfidf_vect_fit.transform(X_test['Document'])len(tfidf_vect_fit.get_feature_names())## 3740tfidf_vect_fit.get_feature_names()[200:350]## ['adjunct', 'adjust', 'administ', 'administr', 'adminmcb', 'admiss', 'admit', 'adolesc', 'adopt', 'ador', 'adult', 'adulteri', 'adults1', 'advanc', 'advantag', 'advers', 'advertis', 'advic', 'advis', 'advisori', 'advoc', 'aerob', 'affair', 'affect', 'affili', 'affirm', 'afford', 'afraid', 'afterhour', 'afterward', 'age', 'agenc', 'agerel', 'aggress', 'agil', 'ago', 'agre', 'ahead', 'aid', 'ailment', 'aim', 'air', 'airport', 'ajman', 'al', 'alchemist', 'alcohol', 'alcoholsoak', 'alert', 'align', 'alik', 'allerg', 'allergi', 'allevi', 'allianc', 'allison', 'allow', 'almost', 'alon', 'along', 'alongsid', 'alreadi', 'alright', 'also', 'alter', 'altern', 'although', 'altogeth', 'alway', 'alzheim', 'amateur', 'amaz', 'amazon', 'amazoncom', 'ambul', 'amelior', 'america', 'american', 'ami', 'among', 'amount', 'amplitud', 'amta', 'analges', 'analysi', 'analyz', 'anatomi', 'ancient', 'andor', 'anecdot', 'anemia', 'anesthet', 'angel', 'anger', 'angl', 'angri', 'anhedonia', 'anim', 'aniston', 'ankl', 'ann', 'annal', 'anni', 'announc', 'annual', 'anosognosia', 'anoth', 'answer', 'antibiot', 'anticip', 'antidepress', 'antiinflammatori', 'antivir', 'anxieti', 'anxietyfre', 'anxiou', 'anyon', 'anyth', 'anywher', 'apa', 'apart', 'appeal', 'appear', 'appendix', 'appetit', 'appli', 'applianc', 'applic', 'appoint', 'approach', 'appropri', 'approv', 'approxim', 'apr', 'april', 'apta', 'archiv', 'area', 'areaswer', 'arent', 'aris', 'arm', 'aromatherapi', 'around', 'arquett', 'array', 'arriv', 'arrow', 'art', 'arteri']tfidf_train_vect=pd.concat([X_train[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(tfidf_train.toarray())],axis=1)
tfidf_test_vect=pd.concat([X_test[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(tfidf_test.toarray())],axis=1)tfidf_train_vect.head()## Document body_length ... 3738 3739
## 0 7 Benefits of Massage Therapy\r\n\r\nMassage t... 5948 ... 0.0 0.0
## 1 What Is Cupping Therapy?\r\n\r\n Cupping ty... 4344 ... 0.0 0.0
## 2 \r\n21 Benefits of Chiropractic Adjustments\r\... 1546 ... 0.0 0.0
## 3 ll You Need To Know About Massage Gun\r\n31 Au... 4598 ... 0.0 0.0
## 4 Goodnight everyone, say it back or Ill massag... 66 ... 0.0 0.0
##
## [5 rows x 3745 columns]from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import precision_recall_fscore_support as score
import time
np.random.seed(45678)rf=RandomForestClassifier(n_estimators=150, max_depth=None, n_jobs=-1)
start=time.time()
rf_model=rf.fit(tfidf_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=rf_model.predict(tfidf_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 79 Not Professional Not Professional
## 20 physical therapy benefits physical therapy benefits
## 38 cupping benefits cupping benefits
## 60 cold stone benefits cold stone benefits
## 63 cold stone benefits cold stone benefits
## 73 Not Professional Not Professional
## 17 physical therapy benefits physical therapy benefits
## 59 cold stone benefits cold stone benefits
## 76 Not Professional Not Professional
## 15 massage benefits massage benefits
## 19 massage benefits physical therapy benefits
## 18 massage benefits physical therapy benefits
## 57 cold stone benefits cold stone benefits
## 67 Not Professional Not Professionalfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.8571428571428571print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[4 0 0 0 0]
## [0 4 0 0 0]
## [0 0 1 0 0]
## [0 0 0 1 0]
## [0 0 0 2 2]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 1.00 1.00 1.00 4
## cold stone benefits 1.00 1.00 1.00 4
## cupping benefits 1.00 1.00 1.00 1
## massage benefits 0.33 1.00 0.50 1
## physical therapy benefits 1.00 0.50 0.67 4
##
## accuracy 0.86 14
## macro avg 0.87 0.90 0.83 14
## weighted avg 0.95 0.86 0.87 14gb=GradientBoostingClassifier(n_estimators=150,max_depth=11)
start=time.time()
gb_model=gb.fit(tfidf_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=gb_model.predict(tfidf_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 79 massage benefits Not Professional
## 20 physical therapy benefits physical therapy benefits
## 38 cupping benefits cupping benefits
## 60 cold stone benefits cold stone benefits
## 63 cold stone benefits cold stone benefits
## 73 Not Professional Not Professional
## 17 physical therapy benefits physical therapy benefits
## 59 cold stone benefits cold stone benefits
## 76 Not Professional Not Professional
## 15 massage benefits massage benefits
## 19 massage benefits physical therapy benefits
## 18 physical therapy benefits physical therapy benefits
## 57 cold stone benefits cold stone benefits
## 67 massage benefits Not Professionalfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.7857142857142857print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[2 0 0 2 0]
## [0 4 0 0 0]
## [0 0 1 0 0]
## [0 0 0 1 0]
## [0 0 0 1 3]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 1.00 0.50 0.67 4
## cold stone benefits 1.00 1.00 1.00 4
## cupping benefits 1.00 1.00 1.00 1
## massage benefits 0.25 1.00 0.40 1
## physical therapy benefits 1.00 0.75 0.86 4
##
## accuracy 0.79 14
## macro avg 0.85 0.85 0.78 14
## weighted avg 0.95 0.79 0.82 14N-Grams Vectorization for RFC and GBC
stopwords = nltk.corpus.stopwords.words('english')
ps=nltk.PorterStemmer()
wn=nltk.WordNetLemmatizer()
data=pd.read_csv("benefitsContraindications3.csv", encoding='unicode_escape')
data.columns=['Document','Source','Topic','InternetSearch','Contraindications','RisksSideEffects']def count_punct(text):
count=sum([1 for char in text if char in string.punctuation])
return round(count/(len(text)-text.count(" ")),3)*100data['body_length']=data['Document'].apply(lambda x: len(x)-x.count(" "))
data['punct%']= data['Document'].apply(lambda x: count_punct(x))def clean_text(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+',text)
text=" ".join([ps.stem(word) for word in tokens if word not in stopwords])#unlisted with N-grams vectorization
return text
def lemmatize(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=" ".join([wn.lemmatize(word) for word in tokens if word not in stopwords])#unlisted with N-grams vectorization
#text=[wn.lemmatize(word) for word in tokens if word not in stopwords]#when using count Vectorization its a list
#or else single letters returned.
return text data['Cleaned_text']=data['Document'].apply(lambda x: clean_text(x))
data['Lemmatized']=data['Document'].apply(lambda x: lemmatize(x))
data.head()## Document ... Lemmatized
## 0 Chiropractic adjustments and treatments serve ... ... chiropractic adjustment treatment serve need m...
## 1 \r\nUnitedHealthcare Combats Opioid Crisis wit... ... unitedhealthcare combat opioid crisis nonopio...
## 2 The Safety of Chiropractic Adjustments\r\nBy ... ... safety chiropractic adjustment lana barhum me...
## 3 Advanced Chiropractic Relief: 8 Key Benefits o... ... advanced chiropractic relief 8 key benefit chi...
## 4 Heading to the spa can be a pampering treat, b... ... heading spa pampering treat also huge boost he...
##
## [5 rows x 10 columns]from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(data[['Document','body_length','punct%','Cleaned_text','Lemmatized']],data['Topic'],test_size=0.15)from sklearn.feature_extraction.text import CountVectorizern_gram_vect=CountVectorizer(ngram_range=(1,4))type(X_train['Cleaned_text'])## <class 'pandas.core.series.Series'>X_train['Cleaned_text'].head()## 33 chiropract care back pain articl chiropract c...
## 32 chiropract care back pain articl chiropract c...
## 18 physic therapi chronic pain expect articl phy...
## 84 know need go er coronaviru sometim stay home r...
## 69 avail good massag plu servic sex top nice romant
## Name: Cleaned_text, dtype: objectX_train['Lemmatized'].head()## 33 chiropractic care back pain article chiroprac...
## 32 chiropractic care back pain article chiroprac...
## 18 physical therapy chronic pain expect article ...
## 84 know need go er coronavirus sometimes staying ...
## 69 available good massage plus service sex top ni...
## Name: Lemmatized, dtype: objectn_gram_vect_fit=n_gram_vect.fit(X_train['Cleaned_text'])
n_gram_train=n_gram_vect_fit.transform(X_train['Cleaned_text'])
n_gram_test=n_gram_vect_fit.transform(X_test['Cleaned_text'])len(n_gram_vect_fit.get_feature_names())## 75530print(n_gram_vect_fit.get_feature_names()[200:500])## ['183 patient', '183 patient neck', '183 patient neck pain', '18772228387', '18772228387 alreadi', '18772228387 alreadi benefit', '18772228387 alreadi benefit va', '18999', '18999 amazoncom', '18999 amazoncom damke', '18999 amazoncom damke profession', '18999 damke', '18999 damke current', '18999 damke current sale', '18h', '18h lmassagebodytobodi', '18h lmassagebodytobodi massag', '18h lmassagebodytobodi massag massag', '19', '19 percent', '19 percent spinal', '19 percent spinal imag', '192', '192 peopl', '192 peopl backrel', '192 peopl backrel pain', '1950', '1950 across', '1950 across hospit', '1950 across hospit china', '1950 cup', '1950 cup also', '1950 cup also practic', '1996', '1996 studi', '1996 studi rand', '1996 studi rand corpor', '19999', '19999 amazoncom', '19999 amazoncom 4000', '19999 amazoncom 4000 amazon', '20', '20 2015', '20 2015 therapi', '20 2015 therapi comment', '20 differ', '20 differ adjust', '20 differ adjust speed', '20 minut', '20 minut depend', '20 minut depend natur', '20 minut long', '20 minut long say', '20 year', '20 year without', '20 year without advers', '2000', '2000 3000', '2000 3000 hour', '2000 3000 hour supervis', '2001', '2001 overview', '2001 overview score', '2001 overview score phq9', '2001 studi', '2001 studi publish', '2001 studi publish canadian', '2003', '2003 evalu', '2003 evalu 183', '2003 evalu 183 patient', '2003 score', '2003 score might', '2003 score might proceed', '2005', '2005 studi', '2005 studi publish', '2005 studi publish intern', '2007', '2007 accord', '2007 accord nation', '2007 accord nation health', '2007 articl', '2007 articl chicago', '2007 articl chicago tribun', '2007 like', '2007 like seek', '2007 like seek coach', '2007 stress', '2007 stress also', '2007 stress also lead', '2008', '2008 center', '2008 center diseas', '2008 center diseas control', '2008 grown', '2008 grown becom', '2008 grown becom favorit', '2008 research', '2008 research evalu', '2008 research evalu chiropract', '2009', '2009 diseas', '2009 diseas link', '2009 diseas link inflamm', '2009 found', '2009 found reflect', '2009 found reflect journal', '2009 reduct', '2009 reduct chronic', '2009 reduct chronic pain', '2010', '2010 studi', '2010 studi publish', '2010 studi publish journal', '2011', '2011 cnnmoneycom', '2011 cnnmoneycom gave', '2011 cnnmoneycom gave physic', '2011 number', '2011 number adult', '2011 number adult us', '2011 review', '2011 review 26', '2011 review 26 clinic', '2011 studi', '2011 studi publish', '2011 studi publish annal', '2012', '2012 report', '2012 report publish', '2012 report publish plo', '2012 review', '2012 review studiestrust', '2012 review studiestrust sourc', '2012 well', '2012 well benefit', '2012 well benefit societi', '2013', '2013 2011', '2013 2011 cnnmoneycom', '2013 2011 cnnmoneycom gave', '2013 illustr', '2013 illustr benefit', '2013 illustr benefit journal', '2014', '2014 2024a', '2014 2024a much', '2014 2024a much quicker', '2014 studi', '2014 studi suggest', '2014 studi suggest vibrat', '2015', '2015 journal', '2015 journal tradit', '2015 journal tradit complementari', '2015 report', '2015 report publish', '2015 report publish journal', '2015 review', '2015 review evid', '2015 review evid found', '2015 therapi', '2015 therapi comment', '2015 therapi comment physic', '2015patricia', '2015patricia mayrhof', '2015patricia mayrhof spatechniqu', '2015patricia mayrhof spatechniqu articles2', '2016', '2016 930', '2016 930 robert', '2016 930 robert shmerl', '2016 cold', '2016 cold stone', '2016 cold stone massag', '2016 michael', '2016 michael phelp', '2016 michael phelp perman', '2016 mr', '2016 mr castaneda', '2016 mr castaneda written', '2016 studi', '2016 studi publish', '2016 studi publish journal', '2016 summer', '2016 summer olymp', '2016 summer olymp share', '2016 use', '2016 use cup', '2016 use cup easili', '2017', '2017 2018', '2017 2018 cdc', '2017 2018 cdc report', '2017 2018 legal', '2017 2018 legal battl', '2017 chiropractor', '2017 chiropractor tout', '2017 chiropractor tout treatment', '2017 nba', '2017 nba final', '2017 nba final irv', '2017 studi', '2017 studi found', '2017 studi found structur', '2018', '2018 84000', '2018 84000 topic', '2018 84000 topic physic', '2018 accord', '2018 accord american', '2018 accord american massag', '2018 cdc', '2018 cdc report', '2018 cdc report physic', '2018 find', '2018 find help', '2018 find help comment', '2018 galluppalm', '2018 galluppalm colleg', '2018 galluppalm colleg chiropract', '2018 getti', '2018 getti imag', '2018 getti imag cup', '2018 individu', '2018 individu salari', '2018 individu salari vari', '2018 largest', '2018 largest percentag', '2018 largest percentag work', '2018 legal', '2018 legal battl', '2018 legal battl aim', '2018 studi', '2018 studi led', '2018 studi led dr', '2019', '2019 1011', '2019 1011 articl', '2019 1011 articl base', '2019 1122', '2019 1122 share', '2019 1122 share articl', '2019 342', '2019 342 pm', '2019 342 pm edt', '2019 earli', '2019 earli 2020', '2019 earli 2020 mani', '2019 found', '2019 found patient', '2019 found patient high', '2019 massag', '2019 massag gun', '2019 massag gun one', '2019 physic', '2019 physic therapist', '2019 physic therapist stretch', '2019 puhhhagetti', '2019 puhhhagetti imag', '2019 puhhhagetti imag come', '2019 rich', '2019 rich barnesgetti', '2019 rich barnesgetti imag', '2019 stori', '2019 stori previous', '2019 stori previous publish', '2019 thai', '2019 thai massag', '2019 thai massag full', '2019 unitedhealthcar', '2019 unitedhealthcar uhc', '2019 unitedhealthcar uhc combat', '2020', '2020 2021', '2020 2021 end', '2020 2021 end expans', '2020 beyond', '2020 beyond peopl', '2020 beyond peopl say', '2020 click', '2020 click read', '2020 click read user', '2020 depress', '2020 depress man', '2020 depress man lie', '2020 karen', '2020 karen green', '2020 karen green last', '2020 mani', '2020 mani peopl', '2020 mani peopl start', '2020 may', '2020 may look', '2020 may look like', '2020 orthoped', '2020 orthoped orthoped', '2020 orthoped orthoped home', '2020 spring']n_gram_train_df=pd.concat([X_train[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(n_gram_train.toarray())],axis=1)
n_gram_test_df=pd.concat([X_test[['Document','body_length','punct%','Cleaned_text','Lemmatized']].reset_index(drop=True),pd.DataFrame(n_gram_test.toarray())],axis=1)n_gram_train_df.head()## Document body_length ... 75528 75529
## 0 \r\nChiropractic Care for Back Pain\r\nIn this... 507 ... 0 0
## 1 \r\nChiropractic Care for Back Pain\r\nIn this... 2366 ... 0 0
## 2 \r\nPhysical Therapy for Chronic Pain: What to... 1854 ... 0 0
## 3 How to Know If You Need to Go to the E.R. With... 8779 ... 0 0
## 4 Available now.. good massage plus service sex.... 68 ... 0 0
##
## [5 rows x 75535 columns]from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.metrics import precision_recall_fscore_support as score
import timerf=RandomForestClassifier(n_estimators=150, max_depth=None, n_jobs=-1)
start=time.time()
rf_model=rf.fit(n_gram_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=rf_model.predict(n_gram_test)
end=time.time()
pred_time=(end-start)
prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 3 chiropractic benefits chiropractic benefits
## 41 cupping benefits cupping benefits
## 62 cold stone benefits cold stone benefits
## 48 Not Professional massage gun benefits
## 43 Not Professional cupping benefits
## 70 Not Professional Not Professional
## 25 Not Professional mental health services benefits
## 75 Not Professional Not Professional
## 21 physical therapy benefits physical therapy benefits
## 15 massage benefits massage benefits
## 77 Not Professional Not Professional
## 55 cold stone benefits cold stone benefits
## 11 Not Professional massage benefits
## 36 Not Professional cupping benefitsfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.6428571428571429print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[3 0 0 0 0 0 0 0]
## [0 1 0 0 0 0 0 0]
## [0 0 2 0 0 0 0 0]
## [2 0 0 1 0 0 0 0]
## [1 0 0 0 1 0 0 0]
## [1 0 0 0 0 0 0 0]
## [1 0 0 0 0 0 0 0]
## [0 0 0 0 0 0 0 1]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 0.38 1.00 0.55 3
## chiropractic benefits 1.00 1.00 1.00 1
## cold stone benefits 1.00 1.00 1.00 2
## cupping benefits 1.00 0.33 0.50 3
## massage benefits 1.00 0.50 0.67 2
## massage gun benefits 0.00 0.00 0.00 1
## mental health services benefits 0.00 0.00 0.00 1
## physical therapy benefits 1.00 1.00 1.00 1
##
## accuracy 0.64 14
## macro avg 0.67 0.60 0.59 14
## weighted avg 0.72 0.64 0.60 14
##
##
## C:\Users\m\Anaconda2\envs\python36\lib\site-packages\sklearn\metrics\classification.py:1437: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.
## 'precision', 'predicted', average, warn_for)gb=GradientBoostingClassifier(n_estimators=150,max_depth=11)
start=time.time()
gb_model=gb.fit(n_gram_train,y_train)
end=time.time()
fit_time=(end-start)start=time.time()
y_pred=gb_model.predict(n_gram_test)
end=time.time()
pred_time=(end-start)prd = pd.DataFrame(y_pred)
prd.columns=['Predicted']
prd.index=y_test.index
pred=pd.concat([pd.DataFrame(prd),y_test],axis=1)
print(pred)## Predicted Topic
## 3 chiropractic benefits chiropractic benefits
## 41 cupping benefits cupping benefits
## 62 massage benefits cold stone benefits
## 48 massage gun benefits massage gun benefits
## 43 cupping benefits cupping benefits
## 70 Not Professional Not Professional
## 25 mental health services benefits mental health services benefits
## 75 Not Professional Not Professional
## 21 physical therapy benefits physical therapy benefits
## 15 massage benefits massage benefits
## 77 Not Professional Not Professional
## 55 massage benefits cold stone benefits
## 11 massage benefits massage benefits
## 36 cupping benefits cupping benefitsfrom sklearn.metrics import classification_report, f1_score, accuracy_score, confusion_matrix
print('accuracy', accuracy_score(y_test, y_pred))## accuracy 0.8571428571428571print('confusion matrix\n', confusion_matrix(y_test, y_pred))## confusion matrix
## [[3 0 0 0 0 0 0 0]
## [0 1 0 0 0 0 0 0]
## [0 0 0 0 2 0 0 0]
## [0 0 0 3 0 0 0 0]
## [0 0 0 0 2 0 0 0]
## [0 0 0 0 0 1 0 0]
## [0 0 0 0 0 0 1 0]
## [0 0 0 0 0 0 0 1]]print('(row=expected, col=predicted)')## (row=expected, col=predicted)print(classification_report(y_test, y_pred))## precision recall f1-score support
##
## Not Professional 1.00 1.00 1.00 3
## chiropractic benefits 1.00 1.00 1.00 1
## cold stone benefits 0.00 0.00 0.00 2
## cupping benefits 1.00 1.00 1.00 3
## massage benefits 0.50 1.00 0.67 2
## massage gun benefits 1.00 1.00 1.00 1
## mental health services benefits 1.00 1.00 1.00 1
## physical therapy benefits 1.00 1.00 1.00 1
##
## accuracy 0.86 14
## macro avg 0.81 0.88 0.83 14
## weighted avg 0.79 0.86 0.81 14
##
##
## C:\Users\m\Anaconda2\envs\python36\lib\site-packages\sklearn\metrics\classification.py:1437: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.
## 'precision', 'predicted', average, warn_for)These next two functions only work with the n-grams vectorized and added Lematized and Cleaned_text feature columns to this data, because with n-grams the tokenizing of the text document has to be word joins and not a list, and the count and tf-idf have to be lists. Otherwise, they count single letters and not words. Derived from a regex style.
def clean_text(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+',text)
text=" ".join([ps.stem(word) for word in tokens if word not in stopwords])
return text
def predict_ngramRFC_clean(new_review):
nr=pd.DataFrame([new_review])
nr.columns=['newReview']
nr['clean']=nr['newReview'].apply(lambda x: clean_text(x))
rf=RandomForestClassifier(n_estimators=150, max_depth=None, n_jobs=-1)
n_gram_vect=CountVectorizer(ngram_range=(1,4))
n_gram_vect_fit=n_gram_vect.fit(X_train['Cleaned_text'])
n_gram_train=n_gram_vect_fit.transform(X_train['Cleaned_text'])
n_gram_test=n_gram_vect_fit.transform(nr['clean'])
model = rf.fit(n_gram_train,y_train)
pred=pd.DataFrame(model.predict(n_gram_test))
pred.columns=['Recommended Healthcare Service']
pred.index= ['stemmedRFC:']
print('\n\n',pred)
def lemmatize(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=" ".join([wn.lemmatize(word) for word in tokens if word not in stopwords])
return text
def predict_ngramRFC_lemma(new_review):
nr=pd.DataFrame([new_review])
nr.columns=['newReview']
nr['lemma']=nr['newReview'].apply(lambda x: lemmatize(x))
rf=RandomForestClassifier(n_estimators=150, max_depth=None, n_jobs=-1)
n_gram_vect=CountVectorizer(ngram_range=(1,4))
n_gram_vect_fit=n_gram_vect.fit(X_train['Lemmatized'])
n_gram_train=n_gram_vect_fit.transform(X_train['Lemmatized'])
n_gram_test=n_gram_vect_fit.transform(nr['lemma'])
model = rf.fit(n_gram_train,y_train)
pred=pd.DataFrame(model.predict(n_gram_test))
pred.columns=['Recommended Healthcare Service:']
pred.index= ['lemmatizedRFC:']
print('\n\n',pred)predict_ngramRFC_clean('I need a massage!') ##
##
## Recommended Healthcare Service
## stemmedRFC: Not Professionalpredict_ngramRFC_lemma('I need a massage!')##
##
## Recommended Healthcare Service:
## lemmatizedRFC: Not ProfessionalWhen lemmatizing or stemming the text with the cleaned text where both were cleaned of punctuation and stop words as well as all made lowercase, the results were the same for the short input of, ‘I need a massage!’
Lets try some other inputs.
predict_ngramRFC_clean('I have been working out a lot more than normal and am sore all over. Feels like a car hit me. I can\'t touch my toes to tie my shoes and my neck won\'t turn to the right. Help me.') ##
##
## Recommended Healthcare Service
## stemmedRFC: Not Professionalpredict_ngramRFC_lemma('I have been working out a lot more than normal and am sore all over. Feels like a car hit me. I can\'t touch my toes to tie my shoes and my neck won\'t turn to the right. Help me.')##
##
## Recommended Healthcare Service:
## lemmatizedRFC: Not Professionaldef clean_text(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+',text)
text=" ".join([ps.stem(word) for word in tokens if word not in stopwords])
return text
def predict_ngramGBC_clean(new_review):
nr=pd.DataFrame([new_review])
nr.columns=['newReview']
nr['clean']=nr['newReview'].apply(lambda x: clean_text(x))
gb=GradientBoostingClassifier(n_estimators=150,max_depth=11)
n_gram_vect=CountVectorizer(ngram_range=(1,4))
n_gram_vect_fit=n_gram_vect.fit(X_train['Cleaned_text'])
n_gram_train=n_gram_vect_fit.transform(X_train['Cleaned_text'])
n_gram_test=n_gram_vect_fit.transform(nr['clean'])
model = gb.fit(n_gram_train,y_train)
pred=pd.DataFrame(model.predict(n_gram_test))
pred.columns=['Recommended Healthcare Service']
pred.index= ['stemmedGBC:']
print('\n\n',pred)
def lemmatize(text):
text="".join([word.lower() for word in text if word not in string.punctuation])
tokens=re.split('\W+', text)
text=" ".join([wn.lemmatize(word) for word in tokens if word not in stopwords])
return text
def predict_ngramGBC_lemma(new_review):
nr=pd.DataFrame([new_review])
nr.columns=['newReview']
nr['lemma']=nr['newReview'].apply(lambda x: lemmatize(x))
gb=GradientBoostingClassifier(n_estimators=150,max_depth=11)
n_gram_vect=CountVectorizer(ngram_range=(1,4))
n_gram_vect_fit=n_gram_vect.fit(X_train['Lemmatized'])
n_gram_train=n_gram_vect_fit.transform(X_train['Lemmatized'])
n_gram_test=n_gram_vect_fit.transform(nr['lemma'])
model = gb.fit(n_gram_train,y_train)
pred=pd.DataFrame(model.predict(n_gram_test))
pred.columns=['Recommended Healthcare Service:']
pred.index= ['lemmatizedGBC:']
print('\n\n',pred)predict_ngramGBC_clean('I need a massage!') ##
##
## Recommended Healthcare Service
## stemmedGBC: Not Professionalpredict_ngramGBC_lemma('I need a massage!')##
##
## Recommended Healthcare Service:
## lemmatizedGBC: Not ProfessionalWhen lemmatizing or stemming the text with the cleaned text where both were cleaned of punctuation and stop words as well as all made lowercase, the results were the same for the short input of, ‘I need a massage!’
Lets try some other inputs.
predict_ngramGBC_clean('I have been working out a lot more than normal and am sore all over. Feels like a car hit me. I can\'t touch my toes to tie my shoes and my neck won\'t turn to the right. Help me.') ##
##
## Recommended Healthcare Service
## stemmedGBC: Not Professionalpredict_ngramGBC_lemma('I have been working out a lot more than normal and am sore all over. Feels like a car hit me. I can\'t touch my toes to tie my shoes and my neck won\'t turn to the right. Help me.')##
##
## Recommended Healthcare Service:
## lemmatizedGBC: Not Professional#X_train['Document'][35]
y_train[35]## 'cupping benefits'testing35 = X_train['Document'][35]
type(testing35)## <class 'str'>Lets test our model on this index 35 of the training set, and see if it will predict the true target, cupping benefits.
predict_ngramGBC_clean(testing35)##
##
## Recommended Healthcare Service
## stemmedGBC: cupping benefitspredict_ngramRFC_clean(testing35)##
##
## Recommended Healthcare Service
## stemmedRFC: cupping benefitspredict_ngramGBC_lemma(testing35)##
##
## Recommended Healthcare Service:
## lemmatizedGBC: cupping benefitspredict_ngramRFC_lemma(testing35)##
##
## Recommended Healthcare Service:
## lemmatizedRFC: cupping benefitsThe above test let us realize that the functions are working, because that was taken directly from the training set and the category or class for that input was cupping benefits as these models should predict or recommend correctly.
Here is the large document for testing35 printed out:
testing35## 'Home ? The Many Benefits of Chinese Cupping\r\nThe Many Benefits of Chinese Cupping\r\nThe Many Benefits of Chinese Cupping\r\n\r\nUpdated:12/02/2019\r\n\r\nBy Kathleen Rushall\r\n\r\n?Acupuncture and cupping, more than half of the ills cured,? is a famous Chinese saying, supporting traditional Chinese medicine. Traditional Chinese medicine brings to mind acupuncture and the use of natural herbs as healing remedies. Cupping is a lesser-known treatment that is also part of Oriental medicine, one that can provide an especially pleasant experience. One of the earliest documentations of cupping can be found in the work titled A Handbook of Prescriptions for Emergencies, which was written by a Taoist herbalist by the name of Ge Hong and which dates all the way back to 300 AD. An even earlier Chinese documentation, three thousand years old, recommended cupping for the treatment of pulmonary tuberculosis. In both Eastern and Western cultures, cupping evolved from shamanistic practices that held the belief that illnesses and infirmities can be sucked out of the body. This article details some of the many benefits of Chinese cupping for your body.\r\n\r\nCupping was established as an official therapeutic practice in the 1950s across hospitals in China after research conducted by Chinese and former Soviet Union acupuncturists confirmed cupping?s effectiveness. Prior to the 1950s, cupping had also been practiced as an auxiliary method in traditional Chinese surgery. In recent years cupping has been growing in popularity, with celebrities like Gwyneth Paltrow, Jennifer Aniston, David Arquette, and athlete Michael Phelps drawing public attention to the traditional benefits of Chinese cupping therapy techniques.\r\n\r\nThough news outlets were quick to criticize celebrities chasing the newest medical therapies and techniques, recent studies have shown cupping?s effectiveness in reducing pain intensity and providing positive short-term benefits.\r\nWhat Is Chinese Cupping? What Does it Do for You?\r\n\r\nCupping is the term applied to a technique that uses small glass cups or bamboo jars as suction devices that are placed on the ski to disperse and break up stagnation and congestion by drawing congested blood, energy or other humors to the surface. In dry cupping, the therapist will simply place the suction cups on the skin. In wet cupping, the practitioner will make a small incision on the skin and then apply the suction cup to draw out small amounts of blood.\r\n\r\nThere are several ways that a practitioner can create the suction in the cups. One method involves swabbing rubbing alcohol onto the bottom of the cup, then lighting it and putting the cup immediately against the skin. Suction can also be created by placing an inverted cup over a small flame, or by using an alcohol-soaked cotton pad over an insulating material (like leather) to protect the skin, then lighting the pad and placing an empty cup over the flame to extinguish it. Flames are never used near the skin and are not lit throughout the process of cupping, but rather are a means to create the heat that causes the suction within the small cups.\r\n\r\nOnce the suction has occurred, the cups can be gently moved across the skin (often referred to as ?gliding cupping). Medical massage oils are sometimes applied to improve movement of the glass cups along the skin. The suction in the cups causes the skin and superficial muscle layer to be lightly drawn into the cup. Cupping is much like the inverse of massage ? rather than applying pressure to muscles, it uses gentle pressure to pull them upward. For most patients, this is a particularly relaxing and relieving sensation. Once suctioned, the cups are generally left in place for about ten minutes while the patient relaxes. This is similar to the practice of Tui Na, a traditional Chinese medicine massage technique that targets acupuncture points as well as painful body parts, and is well known to provide relief through pressure.\r\n\r\nThe side effects of cupping are fairly mild. Bruising should be expected, but skin should return to looking normal within 10 days. Other potential side effects include mild discomfort, skin infection, or burns. However, a trained health professional will apply an antibiotic ointment and bandage to prevent an infection.\r\nThe Philosophy Behind Pain and Cupping. What is Cupping Used For?\r\n\r\n?Where there?s stagnation, there will be pain. Remove the stagnation, and you remove the pain.?\r\n\r\nThe old Chinese medical maxim holds that pain results from the congestion, stagnation, and blockage of Qi, or vital energy, vital fluids, lymph, phlegm, and blood. If pain is the essence of disease, then suffering is a result of obstructed or irregular flow in the body. Chinese cupping is therefore a method of breaking up the blockage to restore the body?s natural flow of energy.\r\n\r\n Interested in becoming a certified acupuncture professional? Visit the campus web site nearest you:\r\n\r\n San Diego acupuncture school\r\n\r\n Chicago acupuncture school\r\n\r\n New York acupuncture school\r\n\r\nCupping Combined With Acupuncture\r\n\r\nGenerally, cupping is combined with acupuncture in one treatment, but it can also be used alone. The suction and negative pressure provided by cupping can loosen muscles, encourage blood flow, and sedate the nervous system (which makes it an excellent treatment for high blood pressure). Cupping is used to relieve back and neck pains, stiff muscles, anxiety, fatigue, migraines, rheumatism, and even cellulite. For weight loss and cellulite treatments, oil is first applied to the skin, and then the cups are moved up and down the surrounding area.\r\n\r\nchinese cupping with acupuncture\r\n\r\nLike acupuncture, cupping follows the lines of the meridians. There are five meridian lines on the back, and these are where the cups are usually placed. Using these points, cupping can help to align and relax qi, as well as target more specific maladies. By targeting the meridian channels, cupping strives to ?open? these channels ? the paths through which life energy flows freely throughout the body, through all tissues and organs, thus providing a smoother and more free-flowing qi (life force). Cupping is one of the best deep-tissue therapies available. It is thought to affect tissues up to four inches deep from the external skin. Toxins can be released, blockages can be cleared, and veins and arteries can be refreshed within these four inches of affected materials. Even hands, wrists, legs, and ankles can be ?cupped,? thus applying the healing to specific organs that correlate with these points.\r\nMore Benefits Of Chinese Cupping\r\n\r\nMore benefits of Chinese cupping for the lungs can clear congestion from a common cold or help to control a person?s asthma. In fact, respiratory conditions are one of the most common maladies that cupping is used to relieve.\r\n\r\nCupping?s detoxifying effect on skin and circulatory system is also significant, with a visible improvement in skin color after three to five treatments. Cupping removes toxins and improves blood flow through the veins and arteries. Especially useful for athletes is cupping?s potential to relieve muscle spasms.\r\n\r\nCupping also affects the digestive system. A few benefits include an improved metabolism, relief from constipation, a healthy appetite, and stronger digestion.\r\n\r\nA 2015 report published in the Journal of Traditional and Complementary Medicine noted cupping as an effective alternative method of treating acne, pain, facial paralysis, cervical spondylosis, and herpes zoster.\r\n\r\nAs health practitioners and researchers continue to study the benefits of cupping, this traditional alternative care technique will gain further acceptance and wider practice across holistic healthcare centers in the U.S. as an effective treatment for a wide variety of ailments.'We can see that a user most likely won’t be inputting this large of a text input into the models, so when predicting a class, the models as trees are most likely using the length of the document to predict, because the category not professional only had one line string inputs. We can try this out with some benefits we know are in the documents, and see if something other than ‘not professional’ is recommended. This next string is a snippet of above.
testing = 'improves blood flow through the veins and arteries. Especially useful for athletes is cupping?s potential to relieve muscle spasms.\r\n\r\nCupping also affects the digestive system. A few benefits include an improved metabolism, relief from constipation, a healthy appetite, and stronger digestion.'predict_ngramGBC_clean(testing)##
##
## Recommended Healthcare Service
## stemmedGBC: Not Professionalpredict_ngramRFC_clean(testing)##
##
## Recommended Healthcare Service
## stemmedRFC: Not Professionalpredict_ngramGBC_lemma(testing)##
##
## Recommended Healthcare Service:
## lemmatizedGBC: Not Professionalpredict_ngramRFC_lemma(testing)##
##
## Recommended Healthcare Service:
## lemmatizedRFC: Not ProfessionalEven though the exact words were from a portion of the cupping benefits document, the models still predicted ‘Not Professional’ for the class. This is also using ngrams tokenization so there would need to be a test of the tf-idf and count tokenizations for both of these tree type models. One thing for sure, the class of ‘Not Professional’ needs 100 times its current document length to be accurate to a degree on short user inputs.