The combination of radiation and immunotherapy is currently enjoying unprecedented attention as a treatment strategy for patients with metastatic cancer. Clinical case studies and proof-of-principle clinical trials report on systemic, abscopal responses to the combination of focal irradiation and immunotherapy in patients who were progressing on immunotherapy alone.

 However, individualized treatment plans to optimally exploit the synergy between immunotherapy and radiotherapy remain elusive due to high intra- and inter-patient heterogeneity and a myriad of possible radiation fractionation protocols, immunotherapy agents, and scheduling options.

Integrated mathematical oncology provides tools that could dissect this complexity and contribute to the transition of synergistic radiation and immunotherapy and radiation-induced abscopal effects into the personalized medicine era. To this end, we need to develop tractable, quantitative models based on carefully selected cancer biology and immunology principles. Predictions of such models, calibrated with patient-specific clinical data, need to be validated in prospective clinical trials.

As a first step toward developing quantitative models, we recently developed a mathematical framework to simulate the systemic dissemination of T cells activated in response to focal therapy.

 Model simulations suggest that metastatic sites within individual patients do not participate equally in immune surveillance and thus are likely to exhibit different systemic responses after local irradiation. We hypothesized that such a model could help identify patient-specific radiation treatment targets that have a high likelihood of inducing abscopal effects. Such targeted treatment strategies would be then worthy of validation in a prospective clinical trial. In a subsequent commentary, this model was critically discussed, with a focus on complex biology that was not incorporated in the model.

 Here we discuss the raised concerns in light of the purpose and applicability of the mathematical model. We echo the need for clinical validation.

Clinical immunotherapy trials, especially if combined with adjuvant focal cytotoxic therapies, have generated encouraging results, including evidence of clinical remissions.

 Radiation therapy can synergize with immunotherapies, such as anti-CTLA4 antibodies or FMS-like tyrasine kinase 3 ligands to generate systemic responses outside the radiation field, as observed in a series of seminal studies by Demaria and Formenti.

 Immune activation after radiation-induced immunogenic cell death provides an explanation for the previously believed anecdotal abscopal effect. Clinical case reports of radiation-induced abscopal effects date back to the 1950s

 and are reported after other immune-activating local therapies such as thermotherapies, including hyperthermia, radiofrequency ablation, and cryotherapy.

The FDA approval of multiple immunotherapeutic agents generates both a clinical need and a prime opportunity to explore and exploit radiation and immunotherapy synergy, particularly for patients with metastatic cancer. In a recent proof-of-principle clinical trial combining local radiotherapy and granulocyte-macrophage colony-stimulating factor, 11 of 41 patients exhibited an objective systemic (abscopal) response.

 However, which metastatic sites were irradiated remained a heuristic decision.

Identifying patient-specific treatment targets adds an additional layer of personalization based upon limited clinical and biological data available for decision making (Figure). Unraveling the complex, adaptive tumor—immune system interactions that determine a response to therapy—both locally in the primary tumor and systemically in metastatic disease&mdash;requires nonlinear understanding and analysis of the multifactorial dynamics that govern them. As identified by Demaria and Formenti, more basic and translational research is needed to decrease treatment outcome uncertainty associated with the biological complexity of these interactions.

 Such research should include: 1) best radiation technique and fractionation protocol to induce antitumor immunity, 2) sequencing of immune modulators, 3) radiation and immunotherapy sensitivity of different tumor types in primary or metastatic tissue environments, 4) possible lack of common expression of antigen(s) or neoantigens between the irradiated and unirradiated metastases, and 5) component of the local immune environment, such as the availability and infiltration of dendritic cells.

Figure. Road map of abscopal effect focused clinical decision-making process. Proposed utilization place for T-cell trafficking framework is highlighted in red.

As a complementary and arguably synergistic methodology to in vitro and in vivo models, mathematical models may contribute to translating basic biological principles into clinical decision making.

 We argue that some of the above-listed uncertainties may be addressed with a quantitative modeling approach where actionable data are available. Tractable, quantitative models can be designed, based on fundamental principles of cancer biology and immunology, to isolate and investigate key mechanisms and relationships within inherently complex biological systems. At present, attempting to elucidate the mechanisms that govern patient response to treatment based on a vast wealth of biological knowledge, but only a minimal amount of clinically obtainable patient information, is a near-impossible task. Instead, the tools of mathematical oncology allow us to identify key players in biological processes by using readily obtainable clinical and experimental data and evaluating its role in the broader setting of patient response. Where such data-driven models demonstrate the ability to reproduce or predict actual clinically observed phenomena, the mechanism in question can be assessed in a prospective clinical trial.

Mathematical models of tumor-immune interactions and therapy

 may be informed, parameterized, and calibrated by the experimental and clinical data collected by Demaria and Formenti

 Mathematical models, like all biological modeling systems including cell culture, tissue culture, or orthotopic xenograft models, to name but a few, are simplifications of tumors in the human body. Thus, have inherent weaknesses. The only question of interest is if the models are illuminating and useful.

 Poleszczuk and colleagues recently developed a mathematical model of T-cell trafficking in the circulatory system of the human body.

 Based on physiological blood flow properties and variation in metastatic tumor volumes and anatomic disease distribution, model simulations revealed that metastatic sites within a patient, when treated with localized therapy, may have different abilities to induce systemic responses. This suggests that the metastases to irradiate in each patient mark an additional, previously unappreciated confounding factor for abscopal responses in the clinic. The mathematical model could help identify tumor sites in individual patients that serve as immunogenic hubs. The model was not designed to, nor can it, predict abscopal effects; it may, however, help to identify patient-specific treatment targets that have the highest potential to mediate an abscopal effect.

In a recent commentary to Poleszczuk’s study

, Demaria and Formenti criticize the simplicity of this modeling approach, as it fails to incorporate a myriad of complex biological processes.

 One major argument is that the importance of activated T-cell dissemination patterns in triggering the abscopal effect is dominated by other factors, such as the availability of dendritic cells. Although a larger relative importance of these other factors is undoubtedly possible, this statement lacks conclusive support. Without such backing, the entropy of activated T-cell distribution among metastatic sites could still be the apparent driver of biological reactions in unirradiated metastases, independent of all other factors, despite their relevance to the question at hand. We agree that the model needs clinical validation with both retrospective and prospective data to investigate the generated biological hypotheses and to determine clinical applicability. There is encouraging clinical results suggesting that anatomical distribution of metastases could impact the likelihood of response to checkpoint blockade. In a recent phase II multicenter clinical trial of atezolizumab (Tecentriq) for metastatic urothelial carcinoma, patients with liver or visceral metastases had a lower objective response rate compared with patients without.

Predicting T-cell trafficking based on biological and physiological mechanisms, while certainly only 1 of many biological processes in abscopal responses, is exclusively calculable and predictable from routinely collected noninvasive imaging, independent of the primary tumor site. It remains currently infeasible to access the mutation load or immune infiltration in all metastatic sites in order to profile T-cell repertoires and to analyze intra- and intermetastases antigen heterogeneity prior to focal radiation. Despite recent radiomics advances

, current radiographic assessments cannot predict immunogenic cell death following prospective irradiation, neither in a single tumor nor as a multimetastases comparison. We will extend our mathematical framework2 beyond its current simplicity when corresponding biological and clinical data become available to inform the model.

We applaud Demaria and Formenti for the proposal to generate an annotated registry of abscopal responders to inform, calibrate, and validate mathematical frameworks to help improve the likelihood of triggering abscopal responses. To date, however, there are too few abscopal effect studies with coherent data to create a statistically meaningful database. In addition to suggested anecdotal case reports,

 an ideal patient cohort to populate such an annotated registry comprises patients accrued to a clinical trial who were selected based on unifying clinical features and received the same treatment and longitudinal analysis. It is of equal importance to include nonresponder data in such a registry, as model predictability must also be guaranteed for patients without abscopal responses. A concerted effort must be made to ensure high-value clinical trial data from accrued patients is shareable after trial conclusion. This may bring together clinicians as well as quantitative and life scientists to ultimately advance our understanding of the complex biology underlying abscopal effects and to inform the next generation of clinical trials with available tools to understand and optimally apply synergistic treatment.

Jan Poleszczuk, Rachel Walker, and Heiko Enderling are with the Department of Integrated Mathematical Oncology, H. Lee Moffitt Cancer Center & Research Institute; Eduardo G. Moros and Heiko Enderling are with Department of Radiation Oncology, H. Lee Moffitt Cancer Center & Research Institute; Mayer Fishman is with the Department of Genitourinary Oncology, H. Lee Moffitt Cancer Center & Research Institute; Julie Djeu is with the Department of Immunology, H. Lee Moffitt Cancer Center & Research Institute; Jonathan D. Schoenfeld is with the Department of Radiation Oncology Department of Radiation Oncology, Brigham and Women’s Hospital and Dana-Farber Cancer Center; and Steven Finkelstein is with the NRG Immunotherapy Committee.

Heiko Enderling and Jan Poleszczuk are supported by the Personalized Medicine Award 09-33504-14-05 and 09-33000-15-03 from the DeBartolo Family Personalized Medicine Institute Pilot Research Awards in Personalized Medicine (PRAPM).

Conceptual design: Heiko Enderling, Eduardo G. Moros, Mayer Fishman, Julie Djeu, Jonathan D. Schoenfeld, Steven Finkelstein, Jan Poleszczuk. Manuscript writing and review: Heiko Enderling, Jan Poleszczuk, and Rachel Walker. All authors read and approved the final manuscript.

Send correspondence to: Heiko Enderling, PhD, Moffitt Cancer Center, 12902 USF Magnolia Drive, Tampa, FL 33612; e-mail: heiko.enderling@moffitt.org.

Golden EB, Chhabra A, Chachoua A, Adams S, Donach M, Fenton-Kerimian M, et al. Local radiotherapy and granulocyte-macrophage colony-stimulating factor to generate abscopal responses in patients with metastatic solid tumours: a proof-of-principle trial. Lancet Oncol. 2015 Jul;16(7):795—803.

Poleszczuk J, Luddy KA, Prokopiou S, Robertson-Tessi M, Moros EG, Fishman M, et al. Abscopal benefits of localized radiotherapy depend on activated T cell trafficking and distribution between metastatic lesions. Cancer Res. 2016;76(5):1009—18.

Demaria S, Formenti SC. Can abscopal effects of local radiotherapy be predicted by modeling T cell trafficking? Journal for ImmunoTherapy of Cancer. BioMed Central; 2016;4(1):29.

Finkelstein SE, Iclozan C, Bui MM, Cotter MJ, Ramakrishnan R, Ahmed J, et al. Combination of external beam radiotherapy (EBRT) with intratumoral injection of dendritic cells as neo-adjuvant treatment of high-risk soft tissue sarcoma patients. Int. J. Radiat. Oncol. Biol. Phys. 2012 Feb 1;82(2):924—32. PMCID: PMC4241354.

Crittenden M, Kohrt H, Levy R, Jones J, Camphausen K, Dicker A, et al. Current Clinical Trials Testing Combinations of Immunotherapy and Radiation. Semin Radiat Oncol. 2015 Jan;25(1):54—64. PMCID: PMC4640687.

Pilones KA, Vanpouille-Box C, Demaria S. Combination of Radiotherapy and Immune Checkpoint Inhibitors. Semin Radiat Oncol. 2015 Jan;25(1):28—33.

Demaria S, Ng B, Devitt ML, Babb JS, Kawashima N, Liebes L, et al. Ionizing radiation inhibition of distant untreated tumors (abscopal effect) is immune mediated. Int. J. Radiat. Oncol. Biol. Phys. 2004 Mar 1;58(3):862—70.

Dewan MZ, Galloway AE, Kawashima N, Dewyngaert JK, Babb JS, Formenti SC, et al. Fractionated but not single-dose radiotherapy induces an immune-mediated abscopal effect when combined with anti-CTLA-4 antibody. Clin. Cancer Res. 2009 Sep 1;15(17):5379—88. PMCID: PMC2746048.

Abuodeh Y, Venkat P, Kim S. Systematic review of case reports on the abscopal effect. Current Problems in Cancer. 2016 Jan;40(1):25—37.

Toraya-Brown S, Fiering S. Local tumour hyperthermia as immunotherapy for metastatic cancer. Int J Hyperthermia. Taylor & Francis; 2014 Dec;30(8):531—9. PMCID: PMC4558619.

O'Brien MA, Power DG, Clover AJP, Bird B, Soden DM, Forde PF. Local tumour ablative therapies: opportunities for maximising immune engagement and activation. Biochim. Biophys. Acta. 2014 Dec;1846(2):510—23.

Shafirstein G, Kaufmann Y, Hennings L, Siegel E, Griffin RJ, Novák P, et al. Conductive interstitial thermal therapy (CITT) inhibits recurrence and metastasis in rabbit VX2 carcinoma model. Int J Hyperthermia. 2009;25(6):446—54. PMCID: PMC2885861.

Anderson ARA, Quaranta V. Integrative mathematical oncology. Nat. Rev. Cancer. 2008 Mar;8(3):227—34.

Rockne RC, Rockhill JK, Mrugala MM, Spence AM, Kalet I, Hendrickson K, et al. Predicting the efficacy of radiotherapy in individual glioblastoma patients in vivo: a mathematical modeling approach. Phys Med Biol. IOP Publishing; 2010 Jun 21;55(12):3271—85. PMCID: PMC3786554.

Yankeelov TE, Atuegwu N, Hormuth D, Weis JA, Barnes SL, Miga MI, et al. Clinically relevant modeling of tumor growth and treatment response. Sci Transl Med. 2013 May 29;5(187):187ps9—187ps9. PMCID: PMC3938952.

Prokopiou S, Moros EG, Poleszczuk J, Caudell J, Torres-Roca JF, Latifi K, et al. A proliferation saturation index to predict radiation response and personalize radiotherapy fractionation. Radiat Oncol. 2015;10(1):159. PMCID: PMC4521490.

Walker R, Enderling H. From concept to clinic: Mathematically informed immunotherapy. Current Problems in Cancer. 2016 Jan;40(1):68—83.

Chandra RA, Wilhite TJ, Balboni TA, Alexander BM, Spektor A, Ott PA, et al. A systematic evaluation of abscopal responses following radiotherapy in patients with metastatic melanoma treated with ipilimumab. Oncoimmunology. 2015 Nov;4(11):e1046028. PMCID: PMC4589983.

Box GEP. Robustness in the Strategy of Scientific Model Building. Robustness in Statistics. Elsevier; 1979. p. 201—36.

Gillies RJ, Kinahan PE, Hricak H. Radiomics: Images Are More than Pictures, They Are Data. Radiology. 2016 Feb;278(2):563—77. PMCID: PMC4734157.

Postow MA, Callahan MK, Barker CA, Yamada Y, Yuan J, Kitano S, et al. Immunologic correlates of the abscopal effect in a patient with melanoma. N. Engl. J. Med. 2012 Mar 8;366(10):925—31. PMCID: PMC3345206.

Golden EB, Demaria S, Schiff PB, Chachoua A, Formenti SC. An abscopal response to radiation and ipilimumab in a patient with metastatic non-small cell lung cancer. Cancer Immunol Res. 2013 Dec;1(6):365—72. PMCID: PMC3930458.

Schoenfeld JD, Mahadevan A, Floyd SR, Dyer MA, Catalano PJ, Alexander BM, et al. Ipilmumab and cranial radiation in metastatic melanoma patients: a case series and review. Journal for ImmunoTherapy of Cancer. BioMed Central; 2015;3(1):50. PMCID: PMC4678639.

Articles

The combination of radiation and immunotherapy is currently enjoying unprecedented attention as a treatment strategy for patients with metastatic cancer.

Modeling T-Cell Trafficking to Increase the Likelihood of Radiation-Induced Abscopal Effects

Head and neck cancer patients develop severe, debilitating symptoms during radiation therapy. Ecological momentary assessment (EMA) of patient-reported outcome measures (PROMs), such as pain, may improve provider awareness of morbidity and improve patient engagement and satisfaction in their care. We demonstrated the feasibility of an application (app) for Android and iOS named PrimeMD for four-times-daily monitoring of pain and severe symptoms in head and neck cancer patients.

Patients undergoing radiation therapy for head and neck cancer that owned and could use a smartphone were recruited from 1 academic medical center. Patient demographics were measured at enrollment. Patients entered pain ratings, use of pain medications, and the presence of severe symptoms for the duration of their radiation treatments. At the conclusion of radiotherapy patients gave an exit interview.

24 patients used the app. Mean age was 59; 46% were female, 92% identified as Caucasian. Forty-two percent had oropharyngeal disease, and all were lymph node positive. Median number of radiation fractions was 33 (range 25-35) with median dose of 6461 cGy; median treatment time 45 days (range 35-50). Median days of at least one use of the app was 12.5 (range 4-45). At the exit interview patients’ median satisfaction was 9/10 and usability was rated at 9.5/10.

Patients experiencing pain and severe symptoms from radiotherapy are able to use a smartphone app for real-time monitoring. Adherence may be improved by adjusting alarms to patient preferences and enabling two-way communication with providers.

Head and neck cancer is the sixth-most common cancer worldwide; there is a male preponderance and the median age of diagnosis is in the early 1960s.

 Patients undergoing definitive radiation treatment for locally-advanced head and neck cancer develop severe symptoms during the first 4 weeks of treatment. Radiation mucositis is a debilitating symptom of radiation treatment and affects about 80% of patients; sufferers may become unable to eat within days.

 Approximately half of patients eventually need feeding tube placement to maintain adequate nutrition, and some require hospitalization for supportive care.

Patients are typically assessed by physicians once every 5 fractions while undergoing daily radiation treatment, sometimes allowing their pain to be untreated between assessments. Mobile smartphone or web-based remote monitoring, whereby patients use an application (app) on their personal devices at predetermined or self-selected intervals to report their symptoms, is a potential solution to improve patient engagement.

The use of such patient-reported outcome monitoring (PROM) systems has been associated with improved symptom control and better engagement with care teams.

 One recent randomized control trial that monitored chemotherapy patients’ symptoms in-office as well as via weekly home emails reported improved health-related quality of life as well as fewer emergency department admissions and a longer duration of chemotherapy.

Study setting, selection criteria, and enrollment

Retrospective recall of symptoms is subject to recall bias and availability.

 Recall of pain in particular is modified by time since the event, the variability of pain over time, and peak intensity of pain.

 Ecological momentary assessment (EMA), or the real-time monitoring of symptoms using paper or electronic diaries, has been recognized as an effective method to increase the accuracy of PRO reporting and has successfully been employed in the monitoring of fibromyalgia pain.

 In the current study, we developed and studied the feasibility of use for a native Android and iOS (iPhone) smartphone app with 4-times-daily real-time EMA of mucositis symptoms for patients treated with radiation therapy for head and neck cancer. The app was designed with predetermined 4-times-daily intervals versus ad libitum symptom reporting to better approximate real-time EMA to explore its EMA in this older-skewing population. We hypothesized that this app for EMA symptom monitoring in this group of patients suffering from severe symptoms of radiation treatment would be successful in engaging patients in reporting their symptoms to their radiation oncologist.

Patients were recruited from one academic radiation oncology clinic from 12/2014-10/2015. Patients were eligible if they were over 18 years old, spoke and read English, were undergoing definitive radiation therapy for head and neck cancer for 5-7 weeks, and if they, or their caretakers, considered to be someone providing the majority of care for activities of daily living, owned an Android or iOS smartphone and demonstrated usage of the notification systems built into their phones. Patients were excluded if they were unable to eat and drink at the time of enrollment, as demonstrated by weight loss or clinical volume depletion. The Vanderbilt Institutional Review Board and Scientific Review Committee approved the study. The trial was registered with the National Cancer Institute (NCI-2014-02426).

Demographic and treatment data collection

Figure 1. Screenshots of PrimeMD user interface.

Physician monitoring of patient reported outcomes

Table 1. Demographic Characteristics of Participants.

Disease characteristics and treatment courses

Table 2. Tumor Characteristics of Participants.

EBV indicates Epstein-Barr virus; HPV, human papillomavirus

Patients were prospectively identified by reviewing the CT simulation schedule in clinic and approached during their simulation visit. After screening, patients gave informed consent; the enrolling staff installed the app on the patient’s phone and gave instruction on its use. During enrollment, the relative meaning of the pain scale in the application was correlated to real-life symptom severity.

Patients’ treatment information was abstracted from their medical records by retrospective chart review. Participants’ demographics were collected by a survey filled out on the day of enrollment. Chart review and survey data were entered into REDCap electronic data capture tools hosted at Vanderbilt University.

The app, named PrimeMD, was developed as a native application for iOS and Android. The user interface is demonstrated in Figure 1. The app was programmed to alert the patient via an alarm to enter his or her symptoms —including mucositis, odynophagia, and dysphagia - at 4 predetermined intervals, customized to each patient, daily. Users could voluntarily report at non-programmed intervals. The alarm prompted the patient whether they felt “OK” or “could be better,” an interaction designed to take 5 or fewer seconds. Patients reporting that they “could be better” were prompted for their current pain in 3 categories distilled from the traditional 10-point pain rating system for ease of use in the smartphone environment: “well-controlled,” “moderate,” and “severe.” “Well-controlled” was defined as present, but tolerable and not affecting quality of life. “Moderate” was distracting or affected quality of life. “Severe” pain was debilitating and required urgent attention. The patient was asked whether they had taken their pain medication (referred to by specific name for each patient). The patient was then asked whether the symptoms impaired their ability to eat by choosing “no,” “some,” or “I can’t eat.” A final prompt confirmed the accuracy of information before transmission and identified the next alarm interval.

Patient responses were uploaded to a secure server accessible only by study staff that collated the above responses over time. Patients reporting “severe” pain or “I can’t eat” for 12 hours, or “moderate” pain for 16 hours, would generate an immediate alert email to the monitoring physician to address severe symptoms.

To assess functionality of the app’s design, we collected comments and feedback from a panel of community experts and laypeople convened by the Vanderbilt Community Engagement Studio, a function of the Meharry-Vanderbilt Community Engaged Research Core. Feedback was collected in one 90-minute session facilitated by a Research Core member and the study organizers. The session was summarized by the Research Core and was used to inform development of the app. A questionnaire soliciting comments and suggestions from users was administered at the first visit after the conclusion of radiation therapy.

Responses from the app were recorded and stored remotely on the secure physician’s dashboard server in comma-separated value format and subsequently imported into SPSS (version 23) for summaries and analyses. Demographic and treatment data were exported from REDCap into SPSS for those summaries. Frequency distributions were used to summarize nominal and ordinal data (eg, sex, type of treatment). Means and standard deviations summarized normally distributed continuous data (eg, age), otherwise medians and inter-quartile ranges were used (eg, days of app use, days in treatment). We defined feasibility as measured by patient engagement with the app, with the primary outcome measure being days of use; we also examined patient feedback at the exit interview and questionnaire to assess whether patients subjectively considered the app useful and functional.

Twenty-three patients and 1 caretaker and patient agreed to take part in the study (24 participants total) . The mean age was 58 years (range 41-82), education 14.2 years (SD=2.9), and 11 (46%) of the patients were female. Most of the patients self-identified as Caucasian (22/24, 91.7%). Approximately half of the sample was married (11/24, 45.8%), more than half employed full- or part-time, and 21% stated that they lived in the country (ie, a rural area). Of those who responded, more than half had incomes of >$60,000 (10/18, 55.6%), approximately half had other medical problems besides the head and neck cancer such as hypertension, diabetes mellitus, or obesity (12/23, 52.2%). See Table 1 for additional details.

The primary site of disease was most often the oropharynx (10/24, 41.7%). The most common histology was squamous cell carcinoma (22/24, 91.7%) with HPV expression positive in approximately 24% of patients assessed (5/21). Seventy-five percent of tumors were stage T2 or less (15/20) and all patients with N Stage information (N=20) were lymph node positive. Concurrent chemoradiotherapy was delivered to 88% (21/24) of participants, with a majority receiving carboplatin and/or paclitaxel (see Table 2). The median number of radiation fractions was 33 (range: 25-35) and the median dose to the primary site was 6461cGy (N=20). Median treatment time was 45 days (range 35-50 days).

Data were recorded for 18 (75%) of patients; data for 6 patients were lost in a database error. Data were recorded independently of the alarm, so alarm frequency and true compliance were not measurable. Patients used the app at least once a median of 12.5 days (IQR: 6-33, range: 4-45). On days with at least some use, the median number of app uses per day was 4.0 with a range of 1 to 5.

The exit interview questionnaire was completed by 18 patients. The median score for general satisfaction was 9/10 (range: 6-10). The respondents were universally very satisfied with the instructions for using PrimeMD, (median: 10/10, range: 8-10) as well as with the usability of the app (median: 9.5/10, range: 4-10). Additionally, patients typically that they would strongly recommend the app to other head and neck cancer patients (median: 10/10, range: 4-10). Perceived benefits of PrimeMD included being closely followed, alarms that could help them take their medications, and that app made them think about how they were feeling. Patients liked how easy, simple and quick PrimeMD was. Detriments noted included bugs in the alarm times, that the alarm sound could be annoying, and the alarm going off at inconvenient times.

PROM has successfully been demonstrated in a variety of contexts, including monitoring cancer patients’ pain and chemotherapy.

 In the current study, we examined the feasibility of 4-times-daily real-time EMA using a native smartphone app in an older population of head and neck cancer patients. We measured participants’ engagement with the app by analyzing the length of time they used it and the number of pain reports they submitted. To our knowledge, this is the first study to show the feasibility of EMA of PROMs in this of cancer patients undergoing predictable and dramatic symptomatic changes who may have low technological intelligence. Our study was constrained by its small sample size and loss of data from 6 participants. Our sample was also ethnically and racially homogenous. We were also unable to commit changes to the app after deployment to correct bugs or annoying features.

Participants demonstrated enthusiasm regarding PrimeMD and their symptom reporting, reflected in their high and consistent engagement and positive responses on the feedback survey. Our participants were of median age comparable to that of head and neck cancer sufferers nationwide, but 42% of our participants reported household incomes >$60,000/year, higher than the median of $53,657 in the United States.

 In addition, our population’s educational attainment broadly reflected trends in the United States; it has been suggested that more-educated populations are more likely to engage with PROMs.

Although our participants reported high satisfaction with the app and quoted enthusiasm for it, actual usage did not reflect consistently high engagement and adherence with the app. Patients who do not experience severe mucositis symptoms may have found intensive monitoring to be invasive or. Adherence may also have been impacted by bugs in the app. The frequency of alarms and their potential to annoy may also have impacted adherence. Other groups have reported consistently high longitudinal engagement with a platform for PROMs in diverse disciplines, including oncology practices.

 Notably, both these platforms featured a greater level of possible engagement with patients than simple reporting of pain and symptoms and afforded two-way communication with providers.

Our participants’ enthusiasm and reported satisfaction with the app despite inconsistent or poor adherence are reflective of the desire of patients to better engage with their providers.

 This is consistent with previous reports that the use of PROMs in the clinical setting can improve patient engagement and satisfaction independent of health outcomes.

 As other studies of intensive symptom monitoring in the office setting have demonstrated improvements in some health outcomes, patient-provider interaction and buy-in from staff at all levels is important to improve adherence and to maximize the effects of PROMs as well.

Here we have demonstrated the implementation of four-times-daily ESA for PROMs in a specific population of sufferers of debilitating pain. Our findings suggest that smartphone-based mobile EMA monitoring is feasible for even very-sick populations of older patients, including a smartphone-naïve 82-year old patient, and that those patients welcome such monitoring and engagement with their physicians. For radiation oncologists, the use of such apps could lead to identification of patients with debilitating symptoms and reduce rates of severe pain, poor quality of life, feeding tube placement, and hospitalization. Future iterations of such an app for ESA of PROMs should include increased two-way communication and acknowledgement of engagement with the patient; they should also include greater flexibility in the implementation of daily alarms to avoid patient fatigue with the app in the absence of actionable symptoms.

Patients undergoing radiation therapy for head and neck cancer tend to be older and male

Such patients are willing and able to use a smartphone application that intensively monitors their symptoms

Patients were happy to use our application, and they appreciated the connection with their providers and participating in their own care

It is particularly important to design intensive real-time monitoring applications to be usable and unobtrusive for patients to use

Jeffrey M. Friedman and Anthony Cmelak are with the Department of Radiation Oncology, Vanderbilt University Medical Center, Nashville, Tennessee. H. Omer Ikizler is with University of Vermont College of Medicine, Burlington, Vermont. Ryan Ber is with Meharry Medical College, Nashville, Tennessee. Mary S. Dietrich is with the Department of Biostatistics, the Department of Psychiatry and Behavioral Sciences, Vanderbilt University School of Medicine, and the Department of Nursing, Vanderbilt University School of Nursing, Nashville, Tennessee. Sheila Ridner is with Vanderbilt University School of Nursing, Nashville, Tennessee.

 Jeffrey M. Friedman, MD, PhD Hematology Oncology Associates of Central New York 5008 Brittonfield Parkway, Suite 700 East Syracuse, NY 13057 Email: jfriedman@hoacny.com Tel: 315-472-7504

Argiris A, Karamouzis MV, Raben D, Ferris RL. Head and neck cancer. 

 2008;371(9625):1695-1709. doi: 10.1016/S0140-6736(08)60728-X.

Rose-Ped AM, Bellm LA, Epstein JB, Trotti A, Gwede C, Fuchs HJ. Complications of radiation therapy for head and neck cancers. The patient's perspective. 

Mortensen HR1, Overgaard J, Specht L, et al. Prevalence and peak incidence of acute and late normal tissue morbidity in the DAHANCA 6&7 randomised trial with accelerated radiotherapy for head and neck cancer. 

 2012;103(1):69-75. doi: 10.1016/j.radonc.2012.01.002.

Trotti A1, Bellm LA, Epstein JB, et al. Mucositis incidence, severity and associated outcomes in patients with head and neck cancer receiving radiotherapy with or without chemotherapy: a systematic literature review. 

Vlacich G1, Diaz R, Thorpe SW, et al. Intensity-modulated radiation therapy with concurrent carboplatin and paclitaxel for locally advanced head and neck cancer: toxicities and efficacy. 

 2012;17(5):673-681. doi: 10.1634/theoncologist.2011-0396.

Mosa AS, Yoo I, Sheets L. A systematic review of healthcare applications for smartphones. 

 2012;12:67. doi: 10.1186/1472-6947-12-67.

Kotronoulas G1, Kearney N, Maguire R, et al. What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. 

2014;32(14):1480-1501. doi: 10.1200/JCO.2013.53.5948.

Basch E, Deal AM, Kris MG, et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. 

 2016;34(6):557-565. doi: 10.1200/JCO.2015.63.0830.

Smyth JM, Stone AA. Ecological momentary assessment research in behavioral medicine. 

Spook JE, Paulussen T, Kok G, Van Empelen P. Monitoring dietary intake and physical activity electronically: feasibility, usability, and ecological validity of a mobile-based Ecological Momentary Assessment tool. 

 2013;15(9):e214. doi: 10.2196/jmir.2617.

Stone AA, Shiffman S, Schwartz JE, Broderick JE, Hufford MR. Patient compliance with paper and electronic diaries. 

Stone AA, Broderick JE. Real-time data collection for pain: appraisal and current status.

Stone AA, Schwartz JE, Broderick JE, Shiffman SS. Variability of momentary pain predicts recall of weekly pain: a consequence of the peak (or salience) memory heuristic. 

Broderick JE, Schwartz JE, Vikingstad G, Pribbernow M, Grossman S, Stone AA. The accuracy of pain and fatigue items across different reporting periods. 

 2008;139(1):146-57. doi: 10.1016/j.pain.2008.03.024.

Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. 

Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. 

2008;11(2):322-333. doi: 10.1111/j.1524-4733.2007.00231.x.

Garcia-Palacios A, Herrero R, Belmonte MA, et al. Ecological momentary assessment for chronic pain in fibromyalgia using a smartphone: a randomized crossover study. 

Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. 

 2009;42(2):377-381. doi: 10.1016/j.jbi.2008.08.010.

Kearney N, Kidd L, Miller M, et al. Utilising handheld computers to monitor and support patients receiving chemotherapy: results of a UK-based feasibility study. 

Larsen ME, Rowntree J, Young AM, et al. Chemotherapy side-effect management using mobile phones. 

 2008;2008:5152-5155. doi: 10.1109/IEMBS.2008.4650374.

Ruland CM, Andersen T, Jeneson A, et al. Effects of an internet support system to assist cancer patients in reducing symptom distress: a randomized controlled trial. 

. 2013;36(1):6-17. doi: 10.1097/NCC.0b013e31824d90d4.

Stinson JN, Jibb LA, Nguyen C, et al. Development and testing of a multidimensional iPhone pain assessment application for adolescents with cancer. 

Mooney KH, Beck SL, Friedman RH, Farzanfar R. Telephone-linked care for cancer symptom monitoring: a pilot study. 

Mirkovic J, Kaufman DR, Ruland CM. Supporting cancer patients in illness management: usability evaluation of a mobile app. 

 2014;2(3):e33. doi: 10.2196/mhealth.3359.

DeNavas-Walt C, Proctor BD. US Census Bureau, Current population reports, P60-252, Income and Poverty in the United States: 2014, US Government Printing Office, Washington, DC, 2015.

Ryan CL, Bauman K. US Census Bureau, Current population reports, P20-578, Educational attainment in the United States: 2015, US Government Printing Office, Washington, DC, 2015.

Judson TJ, Bennett AV, Rogak LJ, et al. Feasibility of long-term patient self-reporting of toxicities from home via the Internet during routine chemotherapy. 

. 2013;31(20):2580-2585. doi: 10.1200/JCO.2012.47.6804.

Hjollund NH, Larsen LP, Biering K, Johnsen SP, Riiskjær E, Schougaard LM. Use of patient-reported outcome (PRO) measures at group and patient levels: experiences from the generic integrated PRO system, WestChronic. 

“Society for Participatory Medicine National Survey,” 2015. http://participatorymedicine.org/wp-content/uploads/2016/01/SPM-National-Survey-12-2015-Methods-and-Results-1.pdf; or alternatively http://participatorymedicine.org/patients-overwhelmingly-want-partnership-with-their-clinicians/. Accesed 4/17/2017.

Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting.

2013;13:211. doi: 10.1186/1472-6963-13-211.

Boyce MB, Browne JP. Does providing feedback on patient-reported outcomes to healthcare professionals result in better outcomes for patients? A systematic review

. 2013;22(9):2265-2278. doi: 10.1007/s11136-013-0390-0.

A feasibility study of a smartphone application that patients undergoing radiation therapy for head and neck cancer can use to regularly report mucositis symptoms.

April 2017 | Contemporary Radiation Oncology

Modeling T-Cell Trafficking to Increase the Likelihood of Radiation-Induced Abscopal Effects

PrimeMD, a Pain Reporting Smartphone Application for Patients Undergoing Head and Neck Radiation