Publish in this journal
Journal Information
Visits
...
Vol. 222. Issue 1.
Pages 1-12 (January 2022)
Original article
Full text access
Derivation and validation of a risk score for admission to the Intensive Care Unit in patients with COVID-19
Derivación y validación de una puntuación de riesgo de ingreso en la Unidad de Cuidados Intensivos para pacientes con COVID-19
Visits
...
J. Enaa,
Corresponding author
ena_jav@gva.es

Corresponding author.
, J.V. Segura-Herasb, E.M. Fonseca-Aizpuruc, M.L. López-Reboirod, A. Gracia-Gutiérreze, J.A. Martín-Oterinof, A. Martin-Urda Diez-Cansecog, C. Pérez-Garcíah, J.M. Ramos-Rincóni, R. Gómez-Huelgasj, on behalf of the SEMI-COVID-19 working group 1
a Servicio de Medicina Interna, Hospital Marina Baixa, Alicante, Spain
b Instituto Universitario de Investigación «Centro de Investigación Operativa» (CIO), Universidad Miguel Hernández, Alicante, Spain
c Servicio de Medicina Interna, Hospital de Cabueñes, Gijón, Asturias, Spain
d Servicio de Medicina Interna, Hospital Público de Monforte de Lemos, Lugo, Spain
e Servicio de Medicina Interna, Hospital General Defensa, Zaragoza, Spain
f Servicio de Medicina Interna, Complejo Asistencial Universitario de Salamanca, Salamanca, Spain
g Servicio de Medicina Interna, Hospital de Palamós, Girona, Spain
h Servicio de Medicina Interna, Hospital do Salnes, Vilagarcía de Arousa, Pontevedra, Spain
i Departamento de Medicina Clínica, Universidad Miguel Hernández de Elche, Alicante, Spain
j Departamento de Medicina Interna, Hospital Regional de Málaga, Instituto de Investigación Biomédica (IBIMA), Universidad de Málaga, Málaga, Spain
Related content
Rev Clin Esp. 2022;222:42-310.1016/j.rceng.2021.08.001
V. Estrada, J. González del Castillo
Article information
Abstract
Full Text
Bibliography
Download PDF
Statistics
Figures (2)
Tables (4)
Table 1. Characteristics of the study patients.
Table 2. Characteristics of patients who were admitted to the ICU in the derivation and validation cohorts.
Table 3. Independent factors of ICU admission in patients hospitalized with COVID-19 for the derivation cohort. Univariate analysis.
Table 4. Independent predictive factors of ICU admission.
Show moreShow less
Abstract
Background

This work aims to identify and validate a risk scale for admission to intensive care units (ICU) in hospitalized patients with coronavirus disease 2019 (COVID-19).

Methods

We created a derivation rule and a validation rule for ICU admission using data from a national registry of a cohort of patients with confirmed SARS-CoV-2 infection who were admitted between March and August 2020 (N = 16,298). We analyzed the available demographic, clinical, radiological, and laboratory variables recorded at hospital admission. We evaluated the performance of the risk score by estimating the area under the receiver operating characteristic curve (AUROC). Using the β coefficients of the regression model, we developed a score (0–100 points) associated with ICU admission.

Results

The mean age of the patients was 67 years; 57% were men. A total of 1420 (8.7%) patients were admitted to the ICU. The variables independently associated with ICU admission were age, dyspnea, Charlson Comorbidity Index score, neutrophil-to-lymphocyte ratio, lactate dehydrogenase levels, and presence of diffuse infiltrates on a chest X-ray. The model showed an AUROC of 0.780 (CI: 0.763−0.797) in the derivation cohort and an AUROC of 0.734 (CI: 0.708−0.761) in the validation cohort. A score of greater than 75 points was associated with a more than 30% probability of ICU admission while a score of less than 50 points reduced the likelihood of ICU admission to 15%.

Conclusion

A simple prediction score was a useful tool for forecasting the probability of ICU admission with a high degree of precision.

Keywords:
Clinical epidemiology
Critical care
Emergency medicine
Respiratory infection
Viral infection
General linear model
Resumen
Fundamento

Identificar y validar una escala de riesgo de ingreso en las unidades de cuidados intensivos (UCI) en pacientes hospitalizados con enfermedad por coronavirus 2019 (COVID-19).

Métodos

Realizamos una regla de derivación y otra de validación para ingreso en UCI utilizando los datos de un registro nacional de cohortes de pacientes con infección confirmada por SARS-CoV-2 ingresados entre marzo y agosto del año 2020 (N = 16.298). Analizamos variables demográficas, clínicas, radiológicas y de laboratorio disponibles en el ingreso hospitalario. Evaluamos el rendimiento de la escala de riesgo mediante estimación del área bajo la curva de característica operativa del receptor (AROC). Utilizamos los coeficientes β del modelo de regresión para elaborar una puntuación (0 a 100 puntos) asociada con ingreso en UCI.

Resultados

La edad media de los pacientes fue 67 años; 57% varones. Un total de 1.420 (8,7%) pacientes ingresaron en la UCI. Las variables independientes asociadas con el ingreso en UCI fueron: edad, disnea, índice de comorbilidad de Charlson, cociente neutrófilos-linfocitos, lactato deshidrogenasa e infiltrados difusos en la radiografia de tórax. El modelo mostró un AROC de 0,780 (IC: 0,763−0,797) en la cohorte de derivación y un AROC de 0,734 (IC: 0,708−0,761) en la cohorte de validación. Una puntuación > 75 se asoció con una probabilidad de ingreso en UCI superior a un 30%, mientras que una puntuación < 50 redujo la probabilidad de ingreso en UCI al 15%.

Conclusión

Una puntuación de predicción simple proporcionó una herramienta útil para predecir la probabilidad de ingreso en la UCI con un alto grado de precisión.

Palabras clave:
Epidemiologia clínica
Urgencias
Unidad de Cuidados Intensivos
Infección respiratoria
Infección viral
Modelo lineal generalizado
Full Text
Introduction

As of April 16, 2021, the number of COVID-19 cases caused by SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) in Spain was 3,407,283. Of them, 340,130 (9.98%) required hospital admission and 31,054 (0.91%) were admitted to intensive care units (ICU)1.

The clinical spectrum of SARS-CoV-2 infection varies from mild symptoms of fever and cough followed by expectoration and fatigue to severe symptoms in critical patients, including sepsis, coagulopathy, respiratory failure, and onset of severe acute respiratory distress syndrome2.

Cumulative mortality in Spain as of April 16, 2021 was 76,981 (2.25%) patients, but among patients admitted to the ICU, mortality was 31%3. Various studies concur that the risk factors associated with mortality include advanced age, presence of kidney failure, low oxygen saturation levels, and high C-reactive protein values4,5.

Early identification of patients who are going to need critical care may be important for providing more appropriate treatment and optimizing available resources.

The purpose of this study is to use an extensive national registry of patients with COVID-19 in order to develop a risk scale for ICU admission applicable to patients who have just been admitted to the hospital.

MethodsData source

The data source was the SEMI-COVID-19 registry, an ongoing retrospective cohort that includes the majority of patients who were discharged or died following hospitalization for confirmed COVID-19 in 150 Spanish hospitals from March 1, 2020 until September 1, 20206. The sample comprised 16,298 patients.

Outcomes

The outcome evaluated was admission to the ICU, measured from the time of admission to the hospital.

Predictor variables

To develop the prediction rule, we used variables that were routinely available at the time of hospital admission and which have been associated with ICU admission in other studies. These variables included: (1) demographics, age, and sex; (2) comorbidities and Charlson Comorbidity Index; (3) signs or symptoms, including dyspnea, confusion, hemoptysis; (4) laboratory data, oxygen saturation in ambient air, LDH levels, the neutrophil-lymphocyte ratio, and C-reactive protein levels.

Derivation model

In the derivation model, we randomly selected two-thirds (n = 10,865) of the sample. To create the derivation model, we conducted a multiple logistic regression analysis with ICU admission as the primary outcome and the previously described clinical parameters as predictor variables. Using the β coefficients, we created a scoring system that divided patients into different risk categories.

Validation model

The validation model was created using the remaining one-third (n = 5433) of the sample.

Statistical analysis

We followed the recommendations of the TRIPOD guidelines in developing the multivariate analysis model7. Continuous variables are expressed as means and standard deviations. Categorical variables are expressed as frequencies and percentages. The chi-square test was used to compare categorical variables and Student's t-test was used to compare continuous variables between groups. Values of p < .05 were considered statistically significant. A multiple logistic regression analysis was conducted in the derivation sample to estimate probability of ICU admission.

To build the derivation model, variables with a p value lower than .10 on the univariate model were selected. The odds ratio (OR) and 95% confidence intervals (95% CI) were estimated based on the regression coefficients. Various regression models were built and the one that was simplest and most explicative based on the Hosmer-Lemeshow goodness of fit test was selected for application to the validation cohort. Nagelkerke’s R2 was used to estimate the proportion of variation explained by the model.

The model’s final performance was evaluated by means of the receiver operating characteristic (ROC) curve and calculation of the area under the ROC curve. Finally, based on the β coefficients, we created a scoring system for establishing different risk levels for ICU admission.

The statistical analysis was conducted using the free R software, version 4.0.2 (Free Software Foundation, Inc. Boston, MA).

Ethical considerations

Personal data were processed pursuant to Law 14/2007, of July 3, on Biomedical Research; Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016, General Data Protection Regulation; and Organic Law 3/2018, of December 5, on Personal Data Protection and Guarantee of Digital Rights.

The SEMI-COVID-19 Registry was approved by the Research Ethics Committee of the Province of Málaga. The Department of Medicines for Human Use of the Spanish Agency of Medicines and Health Products (AEMPS, for its initials in Spanish), in accordance with the applicable precepts, has designated the study to be a “non-post-authorization observational study.”

ResultsCharacteristics of patients admitted to the ICU

A total of 1420 (8.7%) patients were admitted to the ICU (Table 1). Patients who were admitted to the ICU were younger (mean age 63.3 ± 12.3 years) than those who were admitted to conventional hospitalization wards (67.7 ± 16.4 years) (p < .001). Male sex was associated with greater risk of ICU admission (69.5% vs. 56.3%; P < .001).

Table 1.

Characteristics of the study patients.

Characteristic  No. of data points  ICU admission = No (n = 14,868)  ICU admission = Yes (n = 1,420)  p 
Demographics         
Age, years, mean ± SD  16,298  67.7 ± 16.4  63.3 ± 12.3  <.001 
Male sex—No. (%)  16,288  8367 (56.3)  987 (69.5)  <.001 
Dependence  16,288      <.001 
Independent    12,092 (82.4)  1377 (97.8)   
Partially dependent    1461 (10.0)  26 (1.8)   
Totally dependent    1129 (7.7)  5 (0.4)   
Comorbidity—No. (%)         
Smoker  15,627      <.001 
No    9980 (70.0)  876 (64.3)   
Ex-smoker    3567 (25.0)  402 (29.5)   
Active Smoker    717 (5.0)  85 (6.2)   
Hypertension  16,264  7631 (51.4)  705 (49.7)  .226 
Type 2 diabetes mellitus  16,251  2128 (14.3)  228 (16.1)  .086 
Type 1 diabetes mellitus  16,256  817 (5.5)  65 (4.6)  .159 
Chronic kidney disease  16,253  936 (6.3)  47 (3.3)  <.001 
Obesity (BMI > 30 kg/m2)  14,973  885 (21.1)  404 (30.5)  <.001 
Dementia  16,288  1594 (10.7)  3 (0.2)  <.001 
Cancer  16,221  660 (4.5)  43 (3.0)  .015 
COPD  16,263  1037 (7.0)  78 (5.5)  .040 
Chronic liver disease  16,253  156 (1.1)  13 (0.9)  .731 
Chronic heart failure  16,260  1108 (7.5)  43 (3.0)  <.001 
Ischemic heart disease  16,264  846 (5.7)  85 (6.0)  .690 
HIV infection  16,230  85 (0.6)  6 (0.4)  .593 
Signs and symptoms—No. (%)         
Temperature  16,225      <.001 
<37 °C    2418 (16.3)  156 (11.0)   
37−38 °C    3173 (21.4)  223 (15.7)   
>38 °C    9217 (62.2)  1038 (73.3)   
Cough  16,241      <.001 
No    4058 (27.4)  278 (19.6)   
Dry    8519 (57.5)  933 (65.8)   
With sputum production    2246 (15.2)  207 (14.6)   
Anosmia  15,897  1126 (7.8)  68 (4.9)  <.001 
Asthenia  16,090  6377 (43.4)  627 (44.4)  .490 
Anorexia  16,025  2931 (20.0)  262 (18.7)  .249 
Headache  16,072  1765 (12.0)  161 (11.5)  .548 
Arthromyalgia  16,130  4486 (30.5)  473 (33.5)  .019 
Ageusia  15,903  1266 (8.7)  86 (6.2)  .001 
Altered level of consciousness  16,150  1794 (12.2)  119 (8.4)  <.001 
Dyspnea  16,228  8299 (56.0)  1069 (75.4)  <.001 
Vomiting/nausea  16,012  2100 (14.4)  188 (13.3)  .306 
Diarrhea  16,166  3566 (24.2)  321 (22.7)  .217 
Radiology—No. (%)  16,061      <.001 
Absence of infiltrates    7789 (53.1)  630 (44.8)   
Unilateral infiltrates    2567 (17.5)  171 (12.2)   
Diffuse infiltrates    4300 (29.3)  604 (43.0)   
Physical examination, mean ± DE
SpO2, %  15,849  93.3 ± 5.5  89.0 ± 8.8  <.001 
Systolic blood pressure, mmHg  15,666  128.8 ± 21.6  127.8 ± 21.5  .083 
Diastolic blood pressure, mmHg  15,657  73.9 ± 13.3  72.8 ± 13.2  .003 
Heart rate, beats/min  15,831  88.3 ± 17.3  92.8 ± 17.4  <.001 
Tachypnea (<30 breaths/min)  15,942  4258 (29.3)  793 (56.8)  <.001 
Laboratory tests, mean ± SD
Neutrophil-lymphocyte ratio  16,107  7.0 ± 11.6  9.7 ± 9.9  <.001 
Platelets—x109/L  16,182  207,227 ± 93,637  203,388 ± 90,454  .140 
D-dimer—ng/mL  12,852  1,906 ± 10,702  2,969 ± 11,865  .004 
Glucose—mg/dL  15,714  127.5 ± 58.9  138.6 ± 55.2  <.001 
Creatinine—mg/dL  16,151  1.1 ± 0.9  1.1 ± 0.7  .881 
ALT—U/L  15,238  40.5 ± 62.0  49.7 ± 51.0  <.001 
Lactate dehydrogenase—U/L  14,112  359.5 ± 197.1  501.5 ± 365.0  <.001 
C-reactive protein—mg/dL  15,652  84.5 ± 86.1  130.7 ± 108.6  <.001 

ALT: alanine aminotransferase; SD: standard deviation; COPD: chronic obstructive pulmonary disease; BMI: body mass index; SpO2: oxygen saturation; ICU: Intensive Care Unit; HIV: human immunodeficiency virus.

There were comorbidities that were associated with greater risk of ICU admission, including obesity, among others, whereas others were associated with a lower frequency of ICU admission, such as chronic kidney disease, COPD, or chronic heart failure. Smoking was a risk factor for ICU admission. A greater proportion of patients who were admitted to the ICU presented with dyspnea at the time of admission (75.4% vs. 56%, p < .001) as well as a lower rate of altered consciousness (8.4% vs. 12.2%, p < .001), a greater frequency of presence of diffuse infiltrates on a chest x-ray on admission than patients who were admitted to the ICU (43% vs. 29.3% p < .001), and lower oxygen saturation levels (SpO2) than patients admitted to conventional hospitalization wards (89% vs. 93.3%, p < .001).

Regarding laboratory variables, patients hospitalized in the ICU had a greater neutrophil-lymphocyte ratio (9.7% vs. 7.0%, p < .001) and higher d-dimer (2,969.8 vs. 1,906.6 ng/mL, p = .004), glucose (138.6 vs. 127.5 mg/dL, p < .001), alanine aminotransferase (ALT) (49.7 vs. 40.5 U/L, p < .001), lactate dehydrogenase (501.5 vs. 359.5 U/L, p < .001), and C-reactive protein (130.7 vs. 84.5 mg/dL, p < .001) values than patients hospitalized in conventional hospitalization wards.

A total of 568 (40%) of patients admitted to the ICU died whereas those among those admitted to conventional hospitalization wards, there were 2851 (19.2%) deaths.

Predictive model

To create the predictive model, we randomly divided the sample into two parts: two-thirds were the derivation cohort and the remaining one-third was the validation cohort. The characteristics of patients in both cohorts are described in Table 2.

Table 2.

Characteristics of patients who were admitted to the ICU in the derivation and validation cohorts.

Variable  Derivation cohort (n = 10,865)  Validation cohort (n = 5,433)  p 
Age, years
> 75  137 (3.46)  210 (9.16)  .517 
<65  483 (10.77)  140 (11.69)   
[6575]  371 (15.37)  79 (4.08)   
Dyspnea
No  230 (4.98)  118 (5.26)  .108 
Yes  758 (12.23)  311 (9.81)   
Charlson Comorbidity Index
529 (10.8)  11 (3.15)  .461 
223 (9.19)  7 (3.74)   
113 (8.09)  25 (6.61)   
47 (6.33)  56 (8.06)   
24 (5.52)  119 (9.45)   
> 4  36 (4.97)  202 (8.29)   
Neutrophil-lymphocyte ratio
<5  326 (5.85)  144 (5.27)  .335 
5−10  350 (10.92)  152 (9.13)   
> 10  308 (15.83)  131 (13.45)   
Lactate dehydrogenase, U/L
<250  85 (3.5)  27 (2.24)  .655 
250−500  442 (8.24)  209 (7.68)   
> 500  318 (19.79)  133 (17.01)   
Altered level of consciousness
No  902 (9.54)  389 (8.14)  .062 
Yes  84 (6.4)  35 (5.83)   
Chest x-ray
Absence of infiltrates  435 (7.77)  195 (6.91)  .197 
Unilateral infiltrates  114 (6.18)  57 (6.39)   
Diffuse infiltrates  432 (13.24)  172 (10.49)   
Neoplasm
No  961 (9.3)  411 (7.92)  .665 
Yes  28 (5.77)  15 (6.88)   
Tachypnea (> 30 breaths/min)
No  409 (5.67)  194 (5.28)  .171 
Yes  567 (16.66)  226 (13.72)   
Urea elevation > 40 mg/dL
No  406 (8.57)  195 (8.16)  .768 
Yes  447 (10.79)  183 (8.86)   
Type 1 diabetes mellitus
No  947 (9.24)  407 (7.94)  .848 
Yes  43 (7.28)  22 (7.56)   
Type 2 diabetes mellitus
No  831 (8.97)  360 (7.78)  .903 
Yes  159 (10.15)  69 (8.75)   

In the derivation cohort, variables that had a greater association with ICU admission on the univariate analysis were age younger than 75 years, a lower Charlson Comorbidity Index, presence of dyspnea or tachypnea, a neutrophil-lymphocyte ratio greater than five, lactate dehydrogenase values greater than 250 U/L, urea values greater than 40 mg/dL, and presence of diffuse infiltrates on the chest x-ray. The presence of cancer (solid tumor, metastases, hematologic tumor) and altered levels of consciousness were factors that reduced the risk of ICU admission (Table 3).

Table 3.

Independent factors of ICU admission in patients hospitalized with COVID-19 for the derivation cohort. Univariate analysis.

Characteristic  Odds ratio  95% confidence interval  p 
Age, years       
> 75  Ref.     
<65  3.368  (2.780, 4.106)  <.001 
[6575]  5.067  (4.147, 6.227)  <.001 
Charlson Comorbidity Index       
> 4  Ref.     
1.116  (0.649, 1.887)  .685 
1.291  (0.828, 2.029)  .263 
1.682  (1.155, 2.506)  .008 
1.935  (1.364, 2.822)  <.001 
2.313  (1.659, 3.328)  <.001 
Neoplasm       
Yes vs. No  0.597  (0.397, 0.863)  .009 
Type 1 diabetes mellitus       
Yes vs. No  0.770  (0.553, 1.045)  .107 
Type 2 diabetes mellitus       
Yes vs. No  1.146  (0.956, 1.367)  .135 
Altered level of consciousness       
Yes vs. No  0.648  (0.511, 0.812)  <.001 
Tachypnea,< 30 breaths/min       
Yes vs. No  3.327  (2.910, 3.808)  <.001 
Dyspnea       
Yes vs. No  2.659  (2.286, 3.104)  <.001 
Neutrophil-lymphocyte ratio       
<5  Ref.     
5−10  1.974  (1.686. 2.311)  <.001 
> 10  3.027  (2.565. 3.571)  <.001 
Lactate dehydrogenase, U/L       
<250  Ref.     
250−500  2.475  (1.963, 3.157)  <.001 
> 500  6.797  (5.326, 8.765)  <.001 
Urea, mg/dL       
< 40  Ref.     
≥ 40  1.290  (1.120, 1.487)  <.001 
Chest x-ray       
Absence of infiltrates  Ref.     
Unilateral infiltrates  0.781  (0.629, 0.964)  .023 
Diffuse infiltrates  1.811  (1.573, 2.085)  <.001 

According to the multivariate analysis, the variables of age, presence of dyspnea, Charlson Comorbidity Index score, neutrophil-lymphocyte ratio, lactate dehydrogenase values, and chest x-ray data on admission remained in the model (Table 4).

Table 4.

Independent predictive factors of ICU admission.

Variable  β  Odds ratio (95% confidence interval)  p  Points assigned 
Constant  −6.060    <.001   
Age, years         
> 75    Ref.     
<65  1.461  4.311 (3.410, 5.492)  <.001  23 
65−75  1.789  5.982 (4.720, 7.633)  <.001  28 
Dyspnea         
Yes vs. No  0.805  2.237 (1.874, 2.680)  <.001  13 
Charlson Comorbidity Index, points     
> 4    Ref.     
0.098  1.102 (0.577, 2.057)  .762   
0.384  1.468 (0.869, 2.498)  .153   
0.623  1.864 (1.203, 2.967)  .007 
0.699  2.013 (1.338, 3.128)  .001  10 
0.734  2.084 (1.408, 3.195)  .000  10 
Neutrophil-lymphocyte ratio     
<5    Ref.     
5−10  0.558  1.747 (1.455, 2.097)  <.001 
> 10  1.072  2.921 (2.392, 3.568)  <.001  17 
Lactate dehydrogenase, U/L     
<250    Ref.     
250−500  0.663  1.941 (1.518, 2.510)  <.001  10 
> 500  1.591  4.911 (3.763, 6.471)  <.001  25 
Chest x-ray         
Absence of infiltrates    Ref.     
Unilateral infiltrates  0.071  0.931 (0.728, 1.182)  .565   
Diffuse infiltrates  0.412  1.509 (1.279, 1.781)  <.001 

The p value for goodness of fit for the multivariate model, calculated using the Hosmer-Lemeshow statistic, was 0.154, indicating a good fit. Nagelkerke's R2 value showing the model's explanatory value was 0.196. The evaluation of the predictive model’s performance showed an area under the receiver operator characteristic curve (AUROC) of 0.780 (95% CI 0.763−0.797) (Fig. 1). For a probability of ICU admission of 0.102 (cut-off point), the model showed a sensitivity of 67.8% and a specificity of 74.6% in the derivation sample. In the validation cohort, the AUROC was 0.734 and the model showed a sensitivity of 62.1% and a specificity of 72%.

Figure 1.

Receiver operating characteristic curve for ICU admission in the derivation and validation cohort. The area under the receiver operating characteristic curve was 0.781 in the derivation cohort and 0.747 in the validation cohort. The cut-off point which optimizes sensitivity and specificity was 0.087 in both the derivation and validation cohort.

(0.13MB).
Building the scoring system

We built a scoring system based on the β coefficients estimated in the logistic regression model (Table 4). The total individual score (range from 0 to 100 points) was obtained by adding the points of each patient characteristic. This scoring system allows for estimating probability of ICU admission (positive predictive value) with the data obtained at the time of admission. For scores greater than 75 points, the probability of ICU admission is greater than 30% and for scores lower than 50 points, the probability of ICU admission is less than 15% (Fig. 2).

Figure 2.

Relationship between individual score and probability (positive predictive value) that a patient will be admitted to the ICU.

(0.04MB).
Discussion

We developed and validated a clinical prediction model to identify patients with SARS-CoV-2 infection who will require ICU admission. The predictive model’s performance showed an AUC of 0.780 for a probability of ICU admission of 0.102.

Risk of ICU admission increased progressively according to the number of independent variables, which included age younger than 75 years, presence of dyspnea, Charlson Comorbidity Index score of less than 3, a neutrophil-lymphocyte ratio greater than 5, lactate dehydrogenase values greater than 250 U/L, and presence of diffuse infiltrates on the chest x-ray. Using these variables, which are easily available at the time of the patient’s hospital admission, we built a scoring system that allows for determining the probability of ICU admission. This study allows for anticipating the necessary intensive care resources for patients hospitalized due to COVID-19.

The percentage of patients admitted to the ICU in the derivation cohort was 8.9% and in the validation cohort it was 7.7%. These values are similar to what has been reported in other studies conducted in Spain8. The proportion of hospitalized patients admitted to the ICU in other countries was 16.8% in Italy, 17% in the United Kingdom, 11% in France, 32% in the USA, and 8.2% in China9–13. A higher percentage of patients admitted to the ICU applied to our model would entail an increase in the positive predictive value of the scoring system.

The demographic characteristics of the study population also show differences according to country, with a mean age of around 70 years for studies conducted in Europe and a mean age around 50 years for studies conducted in China. All studies showed the determining factors for ICU admission to be older age and male sex, greater comorbidity, and presence of dyspnea. Among the laboratory data, signs of poor prognosis that have been found in all studies include a greater neutrophil-lymphocyte ratio and a greater increase in lactate dehydrogenase levels9–13. Likewise, a diffuse, bilateral pattern with ground glass opacity on the chest x-ray has also been associated with greater risk of severe disease14.

The performance of our predictive model for ICU admission is in line with data found by other authors, who report AUROC values of 0.74 to 0.88 in studies with derivation and validation models12,13.

Nevertheless, we must be cautious when extrapolating the performance of models when applying them to other countries, given that they can have differences that, in some instances, are notable15. These differences can be due to the demographic characteristics of the subjects studied, the incidence of cases at one particular moment in the pandemic, or the structure of the healthcare system, among others.

Among the strengths of this study are the large sample size, which allowed for considering a large number of variables in the predictive model, as well as a robust analysis of the derivation and validation models.

It is advisable to interpret this study as an aid for determining intensive care unit bed requirements for patients with severe COVID-19. Although the risk score refers to clinical deterioration related to COVID-19, the benefit of ICU admission may be conditioned by other aspects. The patient’s comorbidity, presence of advanced cognitive decline, or inability to perform basic activities of daily living can constitute limitations for ICU admission in subjects with severe COVID-1916.

On the other hand, the probability of ICU admission may be influenced by the percentage of beds occupied during different periods of the pandemic. It is possible that during periods when there is good access to supportive treatment, it is decided to provide this treatment to patients with moderate risk of mortality. On the contrary, in periods or areas with more limited resources, the decision could be made to limit intensive supportive treatment in patients with moderate risk in order to optimize available resources. Likewise, the admission of patients with severe COVID-19 in respiratory intensive care units was not taken into consideration; the availability of these facilities could alter the scale validation results.

Finally, the development and validation of the predictive model was carried out in Spain. This could limit the generalization of the scoring system to other areas of the world with different healthcare systems.

In conclusion, we developed a clinical prediction rule for ICU admission for patients with COVID-19. The predictor variables are easy to obtain at the time of admission. Our prediction rule has been validated in a large cohort of patients and has been demonstrated to be reproducible. The incorporation of the rule into the electronic medical record would facilitate its implementation and help clinicians appropriately adapt the patient admissions department.

Funding

J.V. Segura-Heras received a competitive grant from the Ministry of Economy and Competitiveness of Spain, Grant No. MTM2017-83850-P.

The Spanish Society of Internal Medicine (SEMI) is the sponsor of this study. The research has not received specific grants from agencies in the public, commercial, or non-profit sectors.

Conflicts of interest

The authors declare that they have no conflicts of interest.

Acknowledgments

We would like to thank the investigators who participated in data collection for the SEMI-COVID-19 Registry.

Appendix A
List of SEMI-COVID-19 network members

SEMI-COVID-19 Registry Coordinator: José Manuel Casas Rojo.

SEMI-COVID-19 Members of the Scientific Committee: José Manuel Casas Rojo, José Manuel Ramos Rincón, Carlos Lumbreras Bermejo, Jesús Millán Núñez-Cortés, Juan Miguel Antón Santos, Ricardo Gómez Huelgas.

Members of the SEMI-COVID-19 Group

H. U. de Bellvitge. L'Hospitalet de Llobregat, Barcelona: Xavier Corbella, Narcís Homs, Abelardo Montero, José María Mora-Luján, Manuel Rubio-Rivas.

H. U. 12 de Octubre, Madrid: Paloma Agudo de Blas, Coral Arévalo Cañas, Blanca Ayuso, José Bascuñana Morejón, Samara Campos Escudero, María Carnevali Frías, Santiago Cossío Tejido, Borja de Miguel Campo, Carmen Díaz Pedroche, Raquel Díaz Simón, Ana García Reyne, Laura Ibarra Veganzones, Lucía Jorge Huerta, Antonio Lalueza Blanco, Jaime Laureiro Gonzalo, Jaime Lora-Tamayo, Carlos Lumbreras Bermejo, Guillermo Maestro de la Calle, Rodrigo Miranda Godoy, Bárbara Otero Perpiña, Diana Paredes Ruiz, Marcos Sánchez Fernández, Javier Tejada Montes.

H. U. Gregorio Marañón, Madrid: Laura Abarca Casas, Álvaro Alejandre de Oña, Rubén Alonso Beato, Leyre Alonso Gonzalo, Jaime Alonso Muñoz, Christian Mario Amodeo Oblitas, Cristina Ausín García, Marta Bacete Cebrián, Jesús Baltasar Corral, María Barrientos Guerrero, Alejandro D. Bendala Estrada, María Calderón Moreno, Paula Carrascosa Fernández, Raquel Carrillo, Sabela Castañeda Pérez, Eva Cervilla Muñoz, Agustín Diego Chacón Moreno, María Carmen Cuenca Carvajal, Sergio de Santos, Andrés Enríquez Gómez, Eduardo Fernández Carracedo, María Mercedes Ferreiro-Mazón Jenaro, Francisco Galeano Valle, Alejandra García, Irene García Fernández-Bravo, María Eugenia García Leoni, María Gómez Antúnez, Candela González San Narciso, Anthony Alexander Gurjian, Lorena Jiménez Ibáñez, Cristina Lavilla Olleros, Cristina Llamazares Mendo, Sara Luis García, Víctor Mato Jimeno, Clara Millán Nohales, Jesús Millán Núñez-Cortés, Sergio Moragón Ledesma, Antonio Muiño Míguez, Cecilia Muñoz Delgado, Lucía Ordieres Ortega, Susana Pardo Sánchez, Alejandro Parra Virto, María Teresa Pérez Sanz, Blanca Pinilla Llorente, Sandra Piqueras Ruiz, Guillermo Soria Fernández-Llamazares, María Toledano Macías, Neera Toledo Samaniego, Ana Torres do Rego, María Victoria Villalba García, Gracia Villarreal, María Zurita Etayo.

H. Costa del Sol, Marbella, Málaga: Victoria Augustín Bandera, Javier García Alegría, Nicolás Jiménez-García, Jairo Luque del Pino, María Dolores Martín Escalante, Francisco Navarro Romero, Victoria Núñez Rodríguez, Julián Olalla Sierra.

H. de Cabueñes, Gijón, Asturias: Ana María Álvarez Suárez, Carlos Delgado Vergés, Rosa Fernández-Madera Martínez, Eva María Fonseca Aizpuru, Alejandro Gómez Carrasco, Cristina Helguera Amezua, Juan Francisco López Caleya, Diego López Martínez, María del Mar Martínez López, Aleida Martínez Zapico, Carmen Olabuenaga Iscar, Lucía Pérez Casado, María Luisa Taboada Martínez, Lara María Tamargo Chamorro.

H. U. La Paz, Madrid: Jorge Álvarez Troncoso, Francisco Arnalich Fernández, Francisco Blanco Quintana, Carmen Busca Arenzana, Sergio Carrasco Molina, Aranzazu Castellano Candalija, Germán Daroca Bengoa, Alejandro de Gea Grela, Alicia de Lorenzo Hernández, Alejandro Díez Vidal, Carmen Fernández Capitán, María Francisca García Iglesias, Borja González Muñoz, Carmen Rosario Herrero Gil, Juan María Herrero Martínez, Víctor Hontañón, María Jesús Jaras Hernández, Carlos Lahoz, Cristina Marcelo Calvo, Juan Carlos Martín Gutiérrez, Mónica Martínez Prieto, Elena Martínez Robles, Araceli Menéndez Saldaña, Alberto Moreno Fernández, José María Mostaza Prieto, Ana Noblejas Mozo, Carlos Manuel Oñoro López, Esmeralda Palmier Peláez, Marina Palomar Pampyn, María Angustias Quesada Simón, Juan Carlos Ramos Ramos, Luis Ramos Ruperto, Aquilino Sánchez Purificación, Teresa Sancho Bueso, Raquel Sorriguieta Torre, Clara Itziar Soto Abanedes, Yeray Untoria Tabares, Marta Varas Mayoral, Julia Vásquez Manau.

H. Royo Villanova, Zaragoza: Nicolás Alcalá Rivera, Anxela Crestelo Vieitez, Esther del Corral Beamonte, Jesús Díez Manglano, Isabel Fiteni Mera, María del Mar García Andreu, Martin Gericó Aseguinolaza, Cristina Gallego Lezaun, Claudia Josa Laorden, Raúl Martínez Murgui, Marta Teresa Matía Sanz.

H. Reg. U. de Málaga, Málaga: María Mar Ayala-Gutiérrez, Rosa Bernal López, José Bueno Fonseca, Verónica Andrea Buonaiuto, Luis Francisco Caballero Martínez, Lidia Cobos Palacios, Clara Costo Muriel, Francis de Windt, Ana Teresa Fernández-Truchaud Christophel, Paula García Ocaña, Ricardo Gómez Huelgas, Javier Gorospe García, José Antonio Hurtado Oliver, Sergio Jansen-Chaparro, María Dolores López-Carmona, Pablo López Quirantes, Almudena López Sampalo, Elizabeth Lorenzo-Hernández, Juan José Mancebo Sevilla, Jesica Martín Carmona, Luis Miguel Pérez-Belmonte, Iván Pérez de Pedro, Araceli Pineda-Cantero, Carlos Romero Gómez, Michele Ricci, Jaime Sanz Cánovas.

H. Clínico de Santiago de Compostela, A Coruña: María del Carmen Beceiro Abad, María Aurora Freire Romero, Sonia Molinos Castro, Emilio Manuel Paez Guillan, María Pazo Núñez, Paula María Pesqueira Fontán.

H. U. Dr. Peset, Valencia: Juan Alberto Aguilera Ayllón, Arturo Artero, María del Mar Carmona Martín, María José Fabiá Valls, María de Mar Fernández Garcés, Ana Belén Gómez Belda, Ian López Cruz, Manuel Madrazo López, Elisabeth Mateo Sanchís, Jaume Micó Gandía, Laura Piles Roger, Adela María Pina Belmonte, Alba Viana García.

H. Moisès Broggi, Sant Joan Despí, Barcelona: Judit Aranda Lobo, Lucía Feria Casanovas, José Loureiro Amigo, Miguel Martín Fernández, Isabel Oriol Bermúdez, Melani Pestaña Fernández, Nicolás Rhyman, Nuria Vázquez Piqueras.

C. H. U. de Badajoz, Badajoz: Rafael Aragón Lara, Inmaculada Cimadevilla Fernández, Juan Carlos Cira García, Gema María García García, Julia González Granados, Beatriz Guerrero Sánchez, Francisco Javier Monreal Periáñez, María Josefa Pascual Pérez.

H. U. Río Hortega, Valladolid: Irene Arroyo Jiménez, Marina Cazorla González, Marta Cobos-Siles, Luis Corral-Gudino, Pablo Cubero-Morais, María González Fernández, José Pablo Miramontes González, Marina Prieto Dehesa, Pablo Sanz Espinosa.

H. U. Reina Sofía, Córdoba: Antonio Pablo Arenas de Larriva, Pilar Calero Espinal, Javier Delgado Lista, Francisco Fuentes-Jiménez, María del Carmen Guerrero Martínez, María Jesús Gómez Vázquez, José Jiménez Torres, Laura Limia Pérez, José López-Miranda, Laura Martín Piedra, Marta Millán Orge, Javier Pascual Vinagre, Pablo Pérez-Martínez, María Elena Revelles Vílchez, Ángela Rodrigo Martínez, Juan Luis Romero Cabrera, José David Torres-Peña.

H. U. S. Juan de Alicante, Alicante: Marisa Asensio Tomás, David Balaz, David Bonet Tur, Ruth Cañizares Navarro, Paloma Chazarra Pérez, Jesús Corbacho Redondo, Eliana Damonte White, María Escamilla Espínola, Leticia Espinosa Del Barrio, Pedro Jesús Esteve Atiénzar, Carles García Cervera, David Francisco García Núñez, Francisco Garrido Navarro, Vicente Giner Galvañ, Angie Gómez Uranga, Javier Guzmán Martínez, Isidro Hernández Isasi, Lourdes Lajara Villar, Verónica Martínez Sempere, Juan Manuel Núñez Cruz, Sergio Palacios Fernández, Juan Jorge Peris García, Rafael Piñol Pleguezuelos, Andrea Riaño Pérez, José Miguel Seguí Ripoll, Azucena Sempere Mira, Philip Wikman-Jorgensen.

H. Nuestra Señora del Prado, Talavera de la Reina, Toledo: Sonia Casallo Blanco, Jeffrey Oskar Magallanes Gamboa, Cristina Salazar Mosteiro, Andrea Silva Asiain.

H. de Pozoblanco, Córdoba: José Nicolás Alcalá Pedrajas, Antonia Márquez García, Inés Vargas.

H. U. Infanta Cristina, Parla, Madrid: Juan Miguel Antón Santos, Ana Belén Barbero Barrera, Blanca Beamonte Vela, Coralia Bueno Muiño, Charo Burón Fernández, Ruth Calderón Hernáiz, Irene Casado López, José Manuel Casas Rojo, Andrés Cortés Troncoso, Pilar Cubo Romano, Francesco Deodati, Alejandro Estrada Santiago, Gonzalo García Casasola Sánchez, Elena García Guijarro, Francisco Javier García Sánchez, Pilar García de la Torre, Mayte de Guzmán García-Monge, Davide Luordo, María Mateos González, José A. Melero Bermejo, Cruz Pastor Valverde, José Luis Pérez Quero, Fernando Roque Rojas, Lorea Roteta García, Elena Sierra Gonzalo, Francisco Javier Teigell Muñoz, Juan Vicente de la Sota, Javier Villanueva Martínez.

H. G. U. de Elda, Alicante: Carmen Cortés Saavedra, Jennifer Fernández Gómez, Borja González López, María Soledad Hernández Garrido, Ana Isabel López Amorós, Santiago López Gil, María de los Reyes Pascual Pérez, Nuria Ramírez Perea, Andrea Torregrosa García.

H. Santa Marina, Bilbao: María Areses Manrique, Ainara Coduras Erdozain, Ane Labirua-Iturburu Ruiz.

H. San Pedro, Logroño, La Rioja: Diana Alegre González, Irene Ariño Pérez de Zabalza, Sergio Arnedo Hernández, Jorge Collado Sáenz, Beatriz Dendariena, Marta Gómez del Mazo, Iratxe Martínez de Narvajas Urra, Sara Martínez Hernández, Estela Menéndez Fernández, José Luís Peña Somovilla, Elisa Rabadán Pejenaute.

H. U. Son Llàtzer, Palma de Mallorca: Andrés de la Peña Fernández, Almudena Hernández Milián.

C. H. U. Ourense, Ourense: Raquel Fernández González, Amara González Noya, Carlos Hernández Cerón, Isabel Izuzquiza Avanzini, Ana Latorre Diez, Pablo López Mato, Ana María Lorenzo Vizcaya, Daniel Peña Benítez, Milagros María Peña Zemsch, Lucía Pérez Expósito, Marta Pose Bar, Lara Rey González, Laura Rodrigo Lara.

H. U. La Fe, Valencia: Dafne Cabañero, María Calabuig Ballester, Pascual Císcar Fernández, Ricardo Gil Sánchez, Marta Jiménez Escrig, Cristina Marín Amela, Laura Parra Gómez, Carlos Puig Navarro, José Antonio Todolí Parra.

H. de Mataró, Barcelona: Raquel Aranega González, Ramon Boixeda, Javier Fernández Fernández, Carlos Lopera Mármol, Marta Parra Navarro, Ainhoa Rex Guzmán, Aleix Serrallonga Fustier.

H. de Sagunto, Valencia: Enrique Rodilla Sala, José María Pascual Izuel, Zineb Karroud Zamrani.

H. Alto Guadalquivir, Andújar, Jaén: Begoña Cortés Rodríguez.

H. Infanta Margarita, Cabra, Córdoba: María Esther Guisado Espartero, Lorena Montero Rivas, María de la Sierra Navas Alcántara, Raimundo Tirado-Miranda.

C. H. U. de Ferrol, A Coruña: Hortensia Álvarez Díaz, Tamara Dalama López, Estefanía Martul Pego, Carmen Mella Pérez, Ana Pazos Ferro, Sabela Sánchez Trigo, Dolores Suarez Sambade, María Trigas Ferrin, María del Carmen Vázquez Friol, Laura Vilariño Maneiro.

H. Marina Baixa, Villajoyosa, Alicante: Javier Ena, Santiago Pérez Martín.

H. U. Virgen del Rocío, Sevilla: Reyes Aparicio Santos, Máximo Bernabeu-Wittel, Santiago Rodríguez Suárez, María Nieto, Luis Giménez Miranda, Rosa María Gámez Mancera, Fátima Espinosa Torre, Carlos Hernández Quiles, Concepción Conde Guzmán, Juan Delgado de la Cuesta, Jara Eloísa Ternero Vega, María del Carmen López Ríos, Pablo Díaz Jiménez, Bosco Barón Franco, Carlos Jiménez de Juan, Sonia Gutiérrez Rivero, Julia Lanseros Tenllado, Verónica Alfaro Lara, Aurora González Estrada.

H. Público de Monforte de Lemos, Lugo: José López Castro, Manuel Lorenzo López Reboiro, Cristina Sardiña González.

H. General Defensa, Zaragoza: Anyuli Gracia Gutiérrez, Leticia Esther Royo Trallero.

C. A. U. de Salamanca, Salamanca: Gloria María Alonso Claudio, Víctor Barreales Rodríguez, Cristina Carbonell Muñoz, Adela Carpio Pérez, María Victoria Coral Orbes, Daniel Encinas Sánchez, Sandra Inés Revuelta, Miguel Marcos Martín, José Ignacio Martín González, José Ángel Martín Oterino, Leticia Moralejo Alonso, Sonia Peña Balbuena, María Luisa Pérez García, Ana Ramón Prados, Beatriz Rodríguez-Alonso, Ángela Romero Alegría, María Sánchez Ledesma, Rosa Juana Tejera Pérez.

H. de Palamós, Girona: Ana Alberich Conesa, Mari Cruz Almendros Rivas, Miquel Hortos Alsina, José Marchena Romero, Anabel Martín-Urda Diez-Canseco.

H. do Salnes, Vilagarcía de Arousa, Pontevedra: Vanesa Allende Castro, Ana María Baz Lomba, Ruth Brea Aparicio, Marta Fernández Morales, Jesús Manuel Fernández Villar, María Teresa López Monteagudo, Cristina Pérez García, Lorena Rodríguez Ferreira, Diana Sande Llovo, María Begoña Valle Feijoo.

H. U. HM Montepríncipe, Boadilla del Monte, Madrid: José F. Varona Arche.

References
[1]
Secretaría de Estado de Sanidad/Centro de Coordinación de Alertas y Emergencias Sanitarias. Actualización nº 350. Enfermedad por el coronavirus (COVID-19) [Accessed 14 April 2021]. Available from: https://www.mscbs.gob.es/profesionales/saludPublica/ccayes/alertasActual/nCov/documentos/Actualizacion_355_COVID-19.pdf.
[2]
F. Zhou, T. Yu, R. Du, G. Fan, Y. Liu, Z. Liu, et al.
Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study.
Lancet., 395 (2020), pp. 1054-1062
[3]
C. Ferrando, R. Mellado-Artigas, A. Gea, E. Arruti, C. Aldecoa, A. Bordell, et al.
Patient characteristics, clinical course and factors associated to ICU mortality in critically ill patients infected with SARS-CoV-2 in Spain: a prospective, cohort, multicentre study.
Rev Esp Anestesiol Reanim., 67 (2020), pp. 425-437
[4]
J. Berenguer, P. Ryan, J. Rodríguez-Baño, I. Jarrín, J. Carratalà, J. Pachón, et al.
Characteristics and predictors of death among 4035 consecutively hospitalized patients with COVID-19 in Spain.
Clin Microbiol Infect., 26 (2020), pp. 1525-1536
[5]
I.J. Núñez-Gil, C. Fernández-Pérez, V. Estrada, V.M. Becerra-Muñoz, I. El-Battrawy, A. Uribarri, et al.
Mortality risk assessment in Spain and Italy, insights of the HOPE COVID-19 registry.
Intern Emerg Med., 16 (2021), pp. 1-10
[6]
J.M. Casas-Rojo, J.M. Antón-Santos, J. Millán-Núñez-Cortés, C. Lumbreras-Bermejo, J.M. Ramos-Rincón, E. Roy-Vallejo, et al.
Clinical characteristics of patients hospitalized with COVID-19 in Spain: results from the SEMI-COVID-19 Registry.
Rev Clin Esp., 220 (2020), pp. 480-494
[7]
G.S. Collins, J.B. Reitsma, D.G. Altman, K.G. Moons.
Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement.
Ann Intern Med., 162 (2015), pp. 55-63
[8]
C.G. Rodriguez-Gonzalez, E. Chamorro-de-Vega, M. Valerio, M.A. Amor-Garcia, F. Tejerina, M. Sancho-Gonzalez, et al.
COVID-19 in hospitalised patients in Spain: a cohort study in Madrid.
Int J Antimicrob Agents., 57 (2021),
[9]
M. Covino, C. Sandroni, M. Santoro, L. Sabia, B. Simeoni, M.G. Bocci, et al.
Predicting intensive care unit admission and death for COVID-19 patients in the emergency department using early warning scores.
Resuscitation., 156 (2020), pp. 84-91
[10]
A.B. Docherty, E.M. Harrison, C.A. Green, H.E. Hardwick, R. Pius, L. Norman, et al.
Features of 20 133 UK patients in hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study.
[11]
Y. Allenbach, D. Saadoun, G. Maalouf, M. Vieira, A. Hellio, J. Boddaert, et al.
Development of a multivariate prediction model of intensive care unit transfer or death: a French prospective cohort study of hospitalized COVID-19 patients.
[12]
Z. Zhao, A. Chen, W. Hou, J.M. Graham, H. Li, P.S. Richman, et al.
Prediction model and risk scores of ICU admission and mortality in COVID-19.
[13]
W. Liang, H. Liang, L. Ou, B. Chen, A. Chen, C. Li, et al.
Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized patients with COVID-19.
JAMA Intern Med., 180 (2020), pp. 1081-1089
[14]
H. Shi, X. Han, N. Jiang, Y. Cao, O. Alwalid, J. Gu, et al.
Radiological findings from 81 patients with COVID-19 pneumonia in Wuhan, China: a descriptive study.
Lancet Infect Dis., 20 (2020), pp. 425-434
[15]
Ó. Moreno-Pérez, M. Andrés, J.M. León-Ramirez, J. Sánchez-Payá, V. Boix, J. Gil, E. Merino.
The COVID-GRAM tool for patients hospitalized with COVID-19 in Europe.
[16]
B. Guidet, D.W. de Lange, A. Boumendil, S. Leaver, X. Watson, C. Boulanger, et al.
The contribution of frailty, cognition, activity of daily life and comorbidities on outcome in acutely admitted patients over 80 years in European ICUs: the VIP2 study.
Intensive Care Med., 46 (2020), pp. 57-69

Please cite this article as: Ena J, Segura-Heras JV, Fonseca-Aizpuru EM, López-Reboiro ML, Gracia-Gutiérrez A, Martín-Oterino JA, et al. Derivación y validación de una puntuación de riesgo de ingreso en la Unidad de Cuidados Intensivos para pacientes con COVID-19. Rev Clin Esp. 2022;222:1–12.

The full list of members of the SEMI-COVID-19 working group is provided in Appendix A.

Copyright © 2021. Elsevier España, S.L.U. and Sociedad Española de Medicina Interna (SEMI)
Idiomas
Revista Clínica Española (English Edition)

Subscribe to our newsletter

Article options
Tools
es en

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?