J Cancer 2022; 13(6):1758-1767. doi:10.7150/jca.65677 This issue Cite

Research Paper

Nomogram Incorporating Preoperative Testing Markers for the prediction of Early Recurrence for Colorectal Liver Metastases with Neoadjuvant Chemotherapy followed by Hepatectomy

Qichen Chen1#, Yizhou Zhang1#, Xingchen Li2#, Zhen Huang1 Corresponding address, Hong Zhao1 Corresponding address, Jianqiang Cai1

1. Department of Hepatobiliary Surgery, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China.
2. Department of Thoracic Surgery, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China.
#These authors contributed equally to this work.

Citation:
Chen Q, Zhang Y, Li X, Huang Z, Zhao H, Cai J. Nomogram Incorporating Preoperative Testing Markers for the prediction of Early Recurrence for Colorectal Liver Metastases with Neoadjuvant Chemotherapy followed by Hepatectomy. J Cancer 2022; 13(6):1758-1767. doi:10.7150/jca.65677. https://www.jcancer.org/v13p1758.htm
Other styles

File import instruction

Abstract

Graphic abstract

Background: This study aims to develop a nomogram to predict early recurrence in colorectal liver metastases (CRLM) patients who received neoadjuvant chemotherapy (NAC) followed by hepatectomy.

Methods: The primary cohort included 144 CRLM patients treated in the same ward, and 40 CRLM patients from another ward were allocated to the validation cohort. The minimum p value method was used to find the optimal cut-off point for early recurrence. Predictors of early recurrence were determined by univariate analysis and multivariate logistic regression. Nomogram for early recurrence was constructed and area under the receiver operating characteristic curve (AUROC), calibration curves, and decision curve analysis (DCA) were used to evaluate its performance.

Results: The optimal cut-off point for the definition of early recurrence was 11 months. Patients with early recurrence have significantly worse post-recurrence survival (median 23.5 months vs. 35.7 months, P=0.012) and overall survival (median 31.8 months vs. 59.0 months, P<0.001). Preoperative creatinine ≥ 63.5 µmol/L (P=0.035), synchronous liver metastasis (P=0.030), non-R0 resection (P=0.044), postoperative complications (P=0.009) and NAC Cycles ≥ 4 (P=0.003) were independent predictors of early recurrence. A nomogram with favorable discrimination (AUROC of 0.754 in the primary cohort and 0.882 in the validation cohort) and calibration was constructed to predict early recurrence. The DCA results also indicated good clinical applicability.

Conclusions: This is the first study to identify elevated preoperative creatinine as an independent predictor of early recurrence of CRLM treated with hepatectomy after NAC. This study developed a nomogram that particularly incorporated preoperative creatinine to predict early recurrence in CRLM patients receiving NAC to aid clinical decision-making.

Keywords: colorectal liver metastasis, recurrence, neoadjuvant chemotherapy, nomogram

Introduction

Surgical resection is the only treatment that offers potential cure for colorectal liver metastases (CRLM) [1]. A Five-year survival rate of 40-56% can be reached with complete surgical resection [2-6]. However, more than one-half of patients who received hepatectomy for CRLM would develop recurrence, and most recurrences were detected within 2 years after surgery [7-9]. Several studies reported a significant negative impact of early recurrence on the survival of CRLM patients. However, various definitions of early recurrence were used in these studies, with the cut-off value of recurrence-free survival (RFS) ranging from 6 months to 2 years [10-14]. Statistically determined cut-off values of early recurrence have been explored to better demonstrate its impact on survival. Imai et al. determined 8 months as the optimal cut-off value of early recurrence by the minimum p value approach [7]. Liu et al. reported 12 months as the optimal cut-off point for CRLM patients with preoperative chemotherapy [15].

Neoadjuvant chemotherapy (NAC) followed by surgery and adjuvant chemotherapy was an option for initially resectable CRLM [2]. NAC is believed to prolong survival in a subset of patients with a high risk of disease recurrence [16], and improve PFS [2]. NAC may also help prevent patients with disease progression under systematic therapy from receiving futile invasive local therapy [17]. Therefore, the use of NAC on initially resectable CRLM is prevalent.

As early recurrence of CRLM after hepatectomy is associated with significantly worse survival, prediction of early recurrence is crucial. Previous studies identified multiple predictive factors of early recurrence, including patient demographics (e.g. age [7]), clinicopathological characteristics (e.g. bilobular distribution [10], number of metastases [7, 11, 12]), features of intervention (e.g. surgical margin [10, 12]), and cancer-specific biomarkers [e.g. carcinoembryonic antigen (CEA) [10], carbohydrate antigen 19-9 (CA 19-9) [7]. However, none of the previous studies incorporated preoperative blood markers to predict early recurrence. Preoperative blood markers [e.g. gamma-glutamyl transpeptidase (GGT), D-dimer] were easily obtained from routine blood tests and recently found to be novel predictors of the prognosis of CRLM patients [18, 19]. We hypothesized that changes in preoperative blood marker levels could also be predictive of early recurrence. Nomograms are reliable and convenient tools of predicting hazards and facilitate clinical decision-making in CRLM management. Several nomograms were already established to predict the prognosis of CRLM patients with curative surgery after NAC [15, 20-22]. However, these established nomograms focused on predicting postoperative complications or survival, and nomograms focusing on the prediction of early recurrence are lacking. Constructing a nomogram specifically incorporating preoperative blood markers to predict early recurrence may aid clinicians to better distinguish patients of high risk and plan for more personalized postoperative treatment.

This study aims to construct a nomogram incorporating preoperative blood markers to predict early recurrence in CRLM patients with neoadjuvant chemotherapy followed by hepatectomy.

Methods

Study population

This study was approved by our institutional review board and informed consents were obtained from all participants involved in this study. We retrospectively examined the medical records of 513 CRLM patients who received hepatectomy from December 2008 to September 2020, at Cancer Hospital, Chinese Academy of Medical Sciences. The inclusion criteria of this study were: (1) histologically confirmed colorectal adenocarcinoma liver metastasis; (2) received hepatectomy after NAC; (3) developed recurrence. The exclusion criteria of this study were: (1) received neoadjuvant radiotherapy; (2) loss to follow-up; (3) with missing clinical data. Demographics, clinicopathologic features, treatment characteristics, and follow-up information were collected from medical records. Peripheral venous punctures within one week before surgery were used to analyze preoperative blood markers, including GGT, D-dimer, ALT, AST, ALB, FIB, LDH, and creatinine. These preoperative blood markers were included in this study as they were routinely tested and were considered to have a potential association with the prognosis of CRLM patients. This set of preoperative blood markers included in this study was determined before data collection. Synchronous liver metastasis is defined as liver metastasis detected before or at diagnosis of the primary tumor, or during surgery for the primary tumor.

Chemotherapy and Surgical Treatment

As described previously [22], NAC was recommended to patients with one or more factors that are considered to increase the risk of recurrence, including primary lymph node metastasis at diagnosis, synchronous presentation of the primary tumor and liver metastases, and more than one liver lesion. The chemotherapy regimens were FOLFOX (5-flurouracil, leucovorin, and oxaliplatin), FOLFIRI (5-flurouracil, leucovorin, and irinotecan), or XELOX (capecitabine and oxaliplatin), with or without bevacizumab or cetuximab. NAC toxicities included symptoms of the blood system, the digastric tract, or the nervous system. Second-line chemotherapy was given to patients with inadequate control of the disease or severe toxicity during the first-line chemotherapy. Patients who received oxaliplatin in either first-line or second-line chemotherapy were considered as receiving oxaliplatin-based regimen. After the NAC regimens were completed, liver surgeries were performed in four to six weeks. Pathological response to NAC was defined as tumor regression grade (TRG) 1-3 after hepatectomy [23]. Surgical resection of three or more liver segments was defined as major liver resection.

Outcomes and follow-up

Postoperative complications included wound dehiscence, infection (incision site, pulmonary, urinary infection, etc.), anastomotic fistula, hemorrhage, bile leak, ascites, liver failure, urine retention, ileus, atrial fibrillation, and hydrothorax. Postoperative complications were recorded until hospital discharge.

As described previously, patients were followed up at regular intervals after surgery [22]. The first follow-up was done 1 month after hepatectomy. The following follow-ups were done at intervals of 3 months within the first 2 years, 6 months between the second to the fifth year, and once per year thereafter. Recurrence or disease progression was scanned by monitoring level of CEA and imaging. OS and PFS were defined from the date of hepatectomy to death or detection of disease progression, respectively, or to the last available follow-up. Post-recurrence survival (PRS) was calculated from the detection of recurrence to death or to the last available follow-up.

Statistical analysis

The Mann-Whitney U test was used to compare continuous variables, and the Chi-square test or the Fisher's exact test was used to compare categorical factors when considered appropriate. The minimum p value method based on subsequent PRS was used to find the optimal cut-off value to define early recurrence of CRLM. In brief, the study population was categorized into potential early recurrence cohort and potential late recurrence cohort by various cut-off values of RFS with regular intervals. PRS was compared for each evaluated cut-off value by the log-rank test. The cut-off value of RFS with minimum p value in the log-rank test was considered as the optimal cut-off point to define early recurrence. OS and PFS were analyzed by the Kaplan-Meier method and compared by the log-rank test. When converting continuous variables into categorical factors, the cut-off values were determined by the ROC curve, or by clinical relevance. Potential predictors determined by the univariate analysis with P < 0.10 were examined in the multivariate analysis. Multivariate logistic regression was performed to identify independent predictors of early recurrence. Statistical significance was determined as two-tailed P < 0.05. Independent predictors identified in the multivariate analysis were used to construct the nomogram. In brief, the independent predictor with the highest beta coefficient in the logistic regression model was assigned 100 points in the nomogram, and the scores of other independent predictors were assigned as the beta coefficient of the predictor divided by the highest beta coefficient among the predictors [24]. The possibilities of developing early recurrence with various total nomogram scores were calculated from the logistic regression model and plotted on the bottom of the nomogram. In both the primary cohort and the validation cohort, the area under the receiver operating characteristic curve (AUROC) was calculated to evaluate the discrimination of the nomogram. Calibration curves and the Hosmer-Lemeshow test was used to evaluate calibration of the nomogram, and decision curve analysis (DCA) was performed to evaluate the clinical applicability of the nomogram. All statistical analyses were performed by SPSS software version 22 (Armonk NV, USA) and R software (http://www.r-project.org).

Results

Clinicopathological characteristics of patients with recurrence

The primary cohort consists of 144 patients from ward 1 of the Department of Hepatobiliary Surgery in the final study population. Patients in the primary cohort received either staged resection or simultaneous resection. In the case of staged resection, hepatectomy is only performed after complete surgical removal of the primary tumor. One hundred forty-four patients were included in the primary cohort. The median age was 54.5 (IQR: 49.3-61.8) years old in the primary cohort. Sixty-two (43.1%) patients had comorbidities. Seventeen patients (11.8%) had primary tumors in the right hemicolon. One hundred and ten patients (76.4%) had primary lymph node metastases at diagnosis, and 81 patients (56.3%) had bilobular distribution of liver lesions. The patients in the primary cohort had 3 (IQR: 2-5) liver metastases at median, and the median of the largest diameter of liver lesions was 2.8 (IQR: 1.9-4.0) cm. Major liver resection was performed for 122 (84.7%) patients. The median duration of the surgery was 346.5 (IQR: 262.5-423.8) min, and the median volume of intraoperative blood loss was 300 (IQR: 150-500) ml. In this cohort, 21 participants also exhibited extrahepatic metastases.

In the preoperative blood testing, the median level of CEA was 8.1 (IQR: 3.4-36.1) ng/ml in the primary cohort. The median preoperative GGT value was 38.0 (IQR: 24.2-63.8) U/L. The median preoperative D-dimer value was 0.48 (IQR:0.28-0.81) mg/L. The median level of preoperative alanine aminotransferase (ALT) was 20.5 (IQR: 15.0-30.0) U/L. The median level of preoperative aspartate aminotransferase (AST) was 24.0 (IQR: 18.0-29.8) U/L. The median level of preoperative albumin (ALB) was 41.8 (IQR: 38.9-44.2) g/L. The median level of preoperative creatinine was 63.0 (IQR: 56.0-72.8) μmol/L. Oxaliplatin-based NAC regimens were given to 94 (65.3%) patients, and 54 patients (37.5%) received bevacizumab or cetuximab. The median number of cycles of NAC was 5 (IQR: 4-7). NAC toxicities were recorded in 127 patients (88.2%). Fifty-four patients (34.7%) exhibited a favorable pathological response to NAC.

 Figure 1 

Flowchart of the study population

J Cancer Image

We included 40 patients from ward 2 of the Department of Hepatobiliary Surgery in this study as the validation cohort. All patients in the validation cohort received simultaneous resection from January 2018 to September 2020. The flowchart of the study population was shown in Figure 1. Details of demographics and clinicopathological characteristics of the primary cohort and the validation cohort are shown in Table 1.

Defining Early and Late Recurrence

The cut-off thresholds examined for the definition of early recurrence and corresponding RFS, PRS, and OS are shown in Table 2. Based on subsequent PRS, the optimal cut-off threshold of RFS to define early recurrence was 11 months (P=0.012). For patients with early recurrence (<11mo, n=104, 72.2%), the median RFS was 4.0 months (95% CI 2.9-5.1), and the median PRS was 23.5 months (95% CI 19.9-27.1). Patients with late (≥11mo) recurrence (n=40, 27.8%) had a median RFS of 15.4 months (95% CI 14.2-16.0), and a median PRS of 35.7 months (95% CI 9.4-12.2). The median OS of patients with early recurrence was 31.8 months (95% CI 28.7-34.9), and the median OS of patients with late recurrence was 9.0 months (95% CI 37.3-80.8). Patients with early recurrence had significantly unfavorable RFS (P < 0.0001, Figure 2A), PRS (P=0.012, Figure 2B), and OS (P < 0.0001, Figure 2C). When compared to patients with late recurrence, more patients with early recurrence had elevated preoperative creatinine (P=0.047), primary tumor in the right hemicolon (P=0.032), synchronous liver metastasis (P=0.024), more number of liver metastases (P=0.004), non-R0 resection (P=0.041), postoperative complications (P=0.008) and more NAC cycles (P=0.001).

Factors Associated with Early Recurrence

The best cut-off value for preoperative creatinine was 63.5 μmol/L with a sensitivity of 54.8% and specificity of 65.0%. Fifty-seven of 104 patients (54.8%) with preoperative creatinine values exceeding 63.5 μmol/L developed early recurrence, while 14 of 40 patients (35.0%) with preoperative creatinine values less than 63.5 μmol/L had early recurrence (P=0.033). We excluded RAS mutational status from further analyses as this variable was only available in 102 (70.8%) patients of the primary cohort and 29 (72.5%) patients in the validation cohort. The univariate analysis (Table 3) identified preoperative creatinine ≥ 63.5 μmol/L (P=0.035), synchronous liver metastasis (P=0.030), non-R0 resection (P=0.044), postoperative complications (P=0.009), and NAC Cycles ≥ 4 (P=0.003) to be associated with early recurrence. A trend of early recurrence was also observed in patients with the primary tumor in the right hemicolon, with more than one liver metastases, had bilobular distribution of liver metastases, or received radiofrequency ablation (RFA) (P < 0.1).

All potential predictors (P < 0.1) mentioned above were retained in subsequent multivariate analysis. Preoperative creatinine ≥ 63.5 μmol/L (OR = 2.362, 95% CI: 1.027-5.435, P = 0.043), synchronous liver metastasis (OR = 3.416, 95% CI: 1.139-10.251, P = 0.028), postoperative complications (OR =3.261, 95% CI: 1.399-7.599, P = 0.006), and NAC cycles ≥ 4 (OR =3.858, 95% CI: 1.548-9.612, P = 0.004) were independently predictive of early recurrence (Table 3).

 Table 1 

Demographics, Clinicopathologic, and Treatment Characteristics of all patients with Recurrence

VarietyPrimary CohortValidation Cohort
All patients (n=144)Early recurrence (n=104)Late recurrence (n=40)PAll patients (n=40)Early recurrence (n=30)Late recurrence (n=10)P
Demographics and clinicopathological features
Age, median (IQR)54.5 (49.3-61.8)55.0 (50.0-62.0)54.0 (48.3-60.8)0.54758.5 (52.5-63.0)56.5 (53.0-61.8)63.0 (52.3-67.0)0.222
Male, n (%)94 (65.3)70 (67.3)24 (60.0)0.40924 (60.0)20 (66.7)4 (40.0)0.136
BMI≥24kg/m2, n (%)76 (52.8)59 (56.7)17 (42.5)0.12516 (40.0)11 (36.7)5 (50.0)0.456
Comorbidity, n (%)62 (43.1)45 (43.3)17 (42.5)0.93316 (40.0)13 (43.3)3 (30.0)0.456
ASA score 3-4, n (%)19 (13.2)16 (15.4)3 (7.5)0.2106 (15.0)5 (16.7)1 (10.0)0.609
Primary site in colon, n (%)77 (53.5)55 (52.9)22 (55.0)0.82022 (55.0)16 (53.3)6 (60.0)0.714
Right hemicolon, n (%)17 (11.8)16 (15.4)1 (2.5)0.0327 (17.5)3 (10.0)4 (40.0)0.031
Poor differention, n (%)39 (27.1)30 (28.8)9 (22.5)0.44316 (40.0)12 (40.0)4 (40.0)1
T3-T4 stage, n (%)137 (95.1)99 (95.2)38 (95.0)0.96237 (92.5)28 (93.3)9 (90.0)0.729
Primary lymph node metastasis, n (%)110 (76.4)81 (77.9)29 (72.5)0.49632 (80.0)25 (83.3)7 (70.0)0.361
Synchronous liver metastasis, n (%)126 (87.5)95 (91.3)31 (77.5)0.02439 (97.5)29 (96.7)10 (100.0)0.559
Diameter of largest liver metastases, median (IQR)2.8 (1.9-4.0)3.0 (2.0-4.0)2.2 (1.6-4.0)0.2443.0 (1.8-4.5)3.5 (2.4-4.89)1.8 (1.5-2.7)0.020
Number of liver metastases, median (IQR)3.0 (2.0-5.0)4.0 (2.0-5.0)2.0 (1.0-3.0)0.0042.0 (1.0-4.0)2.5 (2.0-4.0)2.0 (1.0-3.5)0.250
Bilobar liver distribution, n (%)81 (56.3)63 (60.6)18 (45.0)0.09121 (52.5)15 (50.0)6 (60.0)0.583
Extrahepatic metastases, n (%)18 (12.5)12 (11.5)6 (15.0)0.5747 (17.5)6 (20.0)1 (10.0)0.471
KRAS mutationa, n (%)30 (20.83)24 (23.1)6 (15.0)0.08615 (51.7)10 (47.6)5 (62.5)0.474
Preoperative laboratory testing levels
Preoperative CEA, median (IQR)8.1 (3.4-36.1)9.1 (3.4-39.4)5.7 (3.4-20.8)0.3207.3 (3.8-21.2)7.3 (4.1-19.6)7.0 (3.2-41.0)0.975
Preoperative GGT, median (IQR)38.0 (24.2-63.8)38.0 (25.0-65.0)33.5 (22.5-58.5)0.55641.0 (29.5-62.0)45.5 (33.0-64.3)36.0 (25.5-43.3)0.206
Preoperative D-dimer, median (IQR)0.48 (0.28-0.81)0.50 (0.28-0.83)0.42 (0.26-0.76)0.4030.35 (0.23-0.62)0.39 (0.22-.61)0.32 (0.28-0.67)0.963
Preoperative ALT, median (IQR)20.5 (15.0-30.0)20.0 (15.0-28.8)21.5 (14.3-32.0)0.59520.0 (13.0-39.3)16.0 (12.3-37.3)29.0 (16.8-57.5)0.122
Preoperative AST, median (IQR)24.0 (18.0-29.8)23.5 (18.0-28.0)25.0 (17.0-31.8)0.48925.0 (19.0-33.3)24.0 (19.0-30.8)32.5 (21.8-52.8)0.067
Preoperative ALB, median (IQR)41.8 (38.9-44.2)41.7 (39.0-44.3)42.0 (36.8-43.9)0.37043.3 (40.6-46.2)43.8 (40.5-46.4)42.5 (41.8-44.5)0.606
Preoperative FIB, median (IQR)3.1 (2.6-3.7)3.1 (2.6-3.7)3.2 (2.8-3.7)0.5683.1 (2.8-3.7)3.0 (2.7-3.7)3.3 (3.0-3.6)0.399
Preoperative LDH, median (IQR)177.5 (150.0-213.8)177.0 (150.0-212.8)180.0 (152.3-217.8)0.808191.0 (161.5-222.0)180.5 (160.5-219.3)206.0 (180.0-283.8)0.349
Preoperative creatinine, median (IQR)63.0 (56.0-72.8)65.0 (58.0-74.8)60.5 (52.3-67.8)0.04766.5 (58.3-75.3)68.0 (56.0-75.5)61.5 (60.0-73.0)0.743
Surgery and chemotherapy details
Heterochronous resection, n (%)40 (27.8)28 (26.9)12 (30.0)0.7120 (0.0)0 (0.0)0 (0.0)NA
R0 resection, n (%)85 (59.0)56 (53.8)29 (72.5)0.04131 (77.5)24 (80.0)7 (70.0)0.512
Major liver resection, n (%)122 (84.7)88 (84.6)34 (85.0)0.95429 (72.5)23 (76.7)6 (60.0)0.307
Concomitant RFA, n (%)33 (22.9)28 (26.9)5 (12.5)0.0651 (2.5)1 (3.3)0 (0.0)0.559
Operation time (min), median (IQR)346.5 (262.5-423.8)349.0 (285.0-431.8)332.5 (238.0-420.0)0.377375.0 (303.8-430.3)395.0 (320.0-452.0)328.0 (252.5-358.8)0.061
Blood loss (ml), median (IQR)300 (150.0-500.0)300.0 (200.0-500.0)200.0 (100.0-400.0)0.186200.0 (200.0-525.0)300.0 (200.0-575.0)200.0 (125.0-275.0)0.162
Postoperative complications, n (%)76 (52.8)62 (59.6)14 (35.0)0.00828 (70.0)22 (73.3)6 (60.0)0.426
Oxaliplatin based NAC regimen, n (%)94 (65.3)64 (61.5)30 (75.0)0.12931 (79.5)22 (73.3)9 (100.0)0.082
NAC cycles, median (IQR)5.0 (4.0-7.0)6.0 (4.0-8.0)4.0 (3.0-6.0)0.0016.0 (3.8-6.3)6.0 (5.3-7.0)3.0 (2.3-4.5)0.002
Preoperative targeted therapy, n (%)54 (37.5)42 (40.4)12 (30.0)0.24928 (70.0)24 (80.0)4 (40.0)0.017
Second-line NAC, n (%)28 (19.4)23 (22.1)5 (12.5)0.19210 (25.0)6 (20.0)4 (40.0)0.206
NAC toxicities, n (%)127 (88.2)91 (87.5)36 (90.0)0.67720 (60.6)15 (57.7)5 (71.4)0.509
Pathological response, n (%)50 (34.7)35 (33.7)15 (37.5)0.66435 (100.0)25 (100.0)10 (100.0)NA
Postoperative chemotherapy, n (%)88 (61.1)62 (59.6)26 (65.0)0.55327 (67.5)22 (73.3)5 (50.0)0.172

a. KRAS status was available in 102 patients of the primary cohort and 29 patients of the validation cohort.

 Figure 2 

Survival analysis of early recurrence vs. late recurrence. A. RFS analysis. B. PRS analysis. C. OS analysis.

J Cancer Image
 Figure 3 

Nomogram predicting the probability of early recurrence in colorectal liver metastases (CRLM) patients.

J Cancer Image
 Table 2 

Evaluated Cut-off Thresholds for defining Early and Late Recurrence based on the Prognosis after Recurrence

Evaluated Cut-offPPotential Early Recurrence CohortPotential Late Recurrence Cohort
NRFS (mo)PRS (mo)OS (mo)NRFS (mo)PRS (mo)OS (mo)
20.742261.223.224.91187.926.735.2
30.179341.42223.61108.62736
40.108491.92223.695102736.8
50.314612.522.224.98310.62736.8
60.183693.022.625.47511.22841
70.114743.022.625.57011.728.341
80.186853.223.530.2591428.842
90.083893.323.530.25514.129.842.3
100.056943.524.330.75014.629.851
110.0121044.023.531.84015.435.751.9
120.0231104.424.3323416-59
130.0351114.424.9323316.6-59
140.1281144.725.332.13016.635.759
150.6861214.92632.12318.22756
160.5591265.12632.11822.82756

Nomogram construction for early recurrence

A nomogram was established with the four independent predictors of early recurrence identified by the multivariate analysis (Figure 3). The specific scores were assigned to each independent predictor as follows: preoperative creatinine ≥ 63.5 μmol/L, 64; synchronous liver metastasis, 91; NAC cycles ≥ 4, 100; and postoperative complications, 88. The cut-off threshold of the nomogram score was set to 217 as determined by the ROC curve. In the primary cohort, the sensitivity of the nomogram was 0.663, and the specificity was 0.725. The nomogram showed acceptable ability to predict early recurrence, with an AUROC value of 0.754 (95% CI: 0.667-0.841, Figure 4A). Calibration curves and the Hosmer-Lemeshow test showed that the nomogram had favorable calibration (chi-square: 1.36, P = 0.998, Figure 4B). The DCA result also indicated that nomograms showed good clinical applicability (Figure 4C). In the validation cohort, high AUROC (0.882, 95% CI: 0.754 - 1) (Figure 5A) was noticed, and good calibration was performed between the predictions by the nomograms and the actual observations (Figure 5B). The DCA result in the validation cohort revealed good clinical applicability (Figure 5C).

 Table 3 

Univariable and Multivariable Logistic Regression for associations between Risk Factors and Early Recurrence of CRLM after Resection (<11 mo)

FactorUnitivariate analysisMultivariate analysis
POR (95%CI) POR (95%CI)
Demographics and clinicopathological features
Age ≥60 years0.7561.133(0.514-2.499)
Male0.4100.729(0.343-1.548)
BMI≥24kg/m20.1281.774(0.849-3.707)
Comorbidity0.9331.032(0.494-2.157)
ASA score 3-40.2202.242(0.616-8.159)
Primary site in colon0.8200.918(0.442-1.910)
Right hemicolon0.0627.091(0.908-55.367)NSNS
Poor differentiation0.4441.396(0.594-3.283)
T3-T4 stage0.9621.042(0.194-5.602)
Primary lymph node metastasis0.4961.336(0.580-3.077)
Synchronous liver metastasis0.0303.065(1.117-8.405)0.0283.416(1.139-10.251)
Diameter of largest liver metastases ≥ 3 cm0.1681.684(0.803-3.531)
Multiple liver metastases0.0672.128(0.950-4.770)NSNS
Bilobar liver distribution0.0941.878(0.899-3.923)NSNS
Extrahepatic metastases0.5750.739(0.257-2.125)
Preoperative laboratory testing levels
Preoperative CEA ≥10 ng/ml0.2691.531(0.719-3.261)
Preoperative GGT ≥ 38 U/L0.6351.194(0.575-2.477)
Preoperative D-dimer ≥0.48 mg/L0.2241.578(0.756-3.295)
Preoperative ALT ≥ 20.5 U/L0.7100.871(0.420-1.807)
Preoperative AST ≥ 24 U/L0.4200.739(0.354-1.542)
Preoperative ALB ≥ 41.8 g/L0.7100.871(0.420-1.807)
Preoperative FIB ≥ 3.15 g/L0.7100.871(0.420-1.807)
Preoperative LDH ≥ 177.5 U/L0.7100.871(0.420-1.807)
Preoperative creatinine ≥ 63.5 μmol/L0.0352.252(1.058-4.796)0.0432.362(1.027-5.435)
Surgery and chemotherapy details
Heterochronous resection0.7120.860(0.385-1.919)
R0 resection0.0440.443(0.200-0.979)NSNS
Major liver resection0.9540.971(0.351-2.687)
Concomitant RFA0.0722.579(0.919-7.241)NSNS
Operation time ≥ 346.5 min0.7101.149(0.554-2.383)
Blood loss ≥ 300 ml0.2291.568(0.753-3.264)
Postoperative complications0.0092.741(1.284-5.854)0.0063.261(1.399-7.599)
Oxaliplatin based NAC regimen0.1320.533(0.235-1.208)
NAC cycles ≥ 40.0033.560(1.534-8.263)0.0043.858(1.548-9.612)
Preoperative targeted therapy0.2511.581(0.723-3.453)
Second-line NAC0.1981.988(0.699-5.653)
NAC toxicities0.6780.778(0.238-2.544)
Pathological response0.6640.845(0.396-1.805)
Postoperative chemotherapy0.5530.795(0.372-1.697)

NS, not significant.

Discussion

Nomogram plays an important part in guiding personalized treatment. It offers a reliable and convenient prediction of prognosis. Several previous studies have established nomograms for the prediction of prognosis of patients with CRLM and underwent NAC. Imai et al. constructed a nomogram to predict OS of patients who presented with initially unresectable liver metastases and received preoperative chemotherapy [20]. Liu et al. created a nomogram to predict disease-free survival (DFS) [15]. The existing nomograms for CRLM mainly focused on the prediction of PFS or OS, while to the best of our knowledge, this is the first study to incorporate preoperative blood markers in the construction of a nomogram focusing on the prediction of early recurrence of CRLM treated with NAC.

In this study, we identified preoperative creatinine ≥ 63.5 μmol/L, synchronous liver metastasis, postoperative complications, and NAC cycles ≥ 4 as four independent predictive factors of early recurrence.

Creatinine is an endogenous biomarker of renal impairment. The levels of serum creatinine can be obtained easily from routine blood tests. The prognostic value of creatinine was confirmed in various malignancies of the urinary system [25, 26], or other systems [27-29]. Elevated serum creatinine is associated with an increased risk of recurrence in colorectal cancer patients [30]. Even with creatinine level within the normal range, the underlying impaired creatinine clearance can be predictive of unfavorable chemotherapy response and time to progression in metastatic colorectal cancer patients with an unspecified site of metastasis [31]. However, evidence of the predictive value of preoperative creatinine on the prognosis of CRLM is lacking. No previous study has reported the prognostic impact of preoperative serum creatinine in patients with CRLM. This is the first study to identify elevated preoperative creatinine as an independent predictor of early recurrence of CRLM treated with hepatectomy after NAC. The correlation between preoperative creatinine and early recurrence can be explained as follows: Elevated creatinine reflects impaired creatinine clearance, which is associated with NAC toxicity. Patients with impaired creatinine clearance who received capecitabine or 5-FU might accumulate their metabolites and may require dose reduction to prevent adverse effects, thereby reducing the efficacy of NAC [32-34]. To date, creatinine is rarely reported in studies investigating prognostic factors of CRLM. As preoperative creatinine can be obtained easily from routine blood tests, further studies should be done to examine its potential predictive value on OS or PFS of CRLM patients.

Synchronous CRLM, accounting for approximately one-half of CRLM cases [1, 6, 12, 35-37], is generally considered to be more aggressive, associated with more disseminated disease stage, and have a worse prognosis compared to CRLM patients with metachronous presentation [1, 36-38]. The worse tumor biology of synchronous CRLM [39] may alter its pattern of recurrence. A greater recurrence rate in the abdominal cavity was observed in patients with synchronous CRLM [38], while several other studies reported synchronous CRLM to be related to worse DFS [40-42]. Hence, it is not surprising that synchronous liver metastasis was identified as an independent predictor of early recurrence, which is in line with a previous study [12].

This study identified postoperative complications as the only surgical factor independently associated with early recurrence. Postoperative complications are known to have a negative impact on the survival and recurrence of CRLM patients [43-45]. In the study of Kaibori et al., postoperative complication was the only independent predictor of early recurrence of CRLM [13]. Severe postoperative complications may prolong the immunosuppression caused by major surgeries, and delay the beginning of adjuvant chemotherapy, which is considered to facilitate disease recurrence [13]. In the present study, the postoperative complication rate reached a relatively high 52.8% (76 in 144), which suggested that better control of postoperative complications may play an important part in the prevention of early recurrence.

NAC cycles ≥ 4 is the only chemotherapy-related factor independently predictive of early recurrence. Patients who received more cycles of NAC might have a less favorable disease stage, poorer response to first-line chemotherapy regimen, and could be involved in more complicated surgeries. Furthermore, prolonged NAC duration may introduce chemotherapy-associated liver injuries [46]. Six or more cycles of preoperative oxaliplatin-based chemotherapy were significantly associated with a higher incidence of sinusoidal injury [47]. The long-term impact of chemotherapy-associated liver injuries remains unclear, but they were considered to decrease survival indirectly through poor tumor response [48]. Of note, the present study did not show a significant correlation between postoperative chemotherapy and early recurrence. The protective effect of adjuvant chemotherapy in CRLM patients treated with hepatectomy after NAC requires further investigation.

 Figure 4 

Evaluation of nomogram in the prediction of early recurrence in the primary cohort. A. The ROC curves of the nomogram. B. The calibration curves for predicting early recurrence presence. C. The DCA analysis.

J Cancer Image
 Figure 5 

Evaluation of nomogram in the prediction of early recurrence in the validation cohort. A. The ROC curves of the nomogram. B. The calibration curves for predicting early recurrence presence. C. The DCA analysis.

J Cancer Image

As nomogram offers a simple graphical presentation of statistical prediction models, it is favored by both clinicians and patients and proved useful for various types of cancer [24]. This nomogram would enable clinicians to conveniently evaluate the risk of early recurrence of individual CRLM patients, and allow personalized adjustment in further treatment planning. Changes in planned adjuvant chemotherapy or monitoring strategy based on the nomogram scores of every single patient can be feasible. Patients with high nomogram scores may require stronger adjuvant chemotherapy and more frequent monitoring. Patients with low nomogram scores may be saved from receiving unnecessary invasive treatment or adjuvant chemotherapy.

This study has several limitations. First, this study is limited by its retrospective design. Causality cannot be inferred confidently in a retrospective observational study due to potential residual confounding. Second, although many factors including preoperative blood markers were incorporated in this study, it is still possible that unmeasured factors would affect early recurrence. Third, the data is collected from a single institution and exposed to potential selection bias. Although the nomogram exhibited acceptable predictability in our internal validation, external validation at other institutions is preferred to strengthen our results. Finally, the cohort may not be of enough size to demonstrate statistical significance for some variables. A prospective study with a larger cohort might resolve these limitations, validate our results and provide stronger evidence.

Conclusion

This is the first study to identify elevated preoperative creatinine as an independent predictor of early recurrence of CRLM treated with hepatectomy after NAC. This study developed a nomogram that particularly incorporated preoperative creatinine to predict early recurrence in CRLM patients receiving NAC. This nomogram exhibits favorable discrimination and calibration to aid clinical decision-making.

Abbreviations

CRLM: colorectal liver metastasis; NAC: neoadjuvant chemotherapy; AUROC: area under the receiver operating characteristics curve; DCA: decision curve analysis; CEA: carcinoembryonic antigen; CA 19-9: carbohydrate antigen 19-9; GGT: gamma-glutamyl transpeptidase; TRG: tumor regression grade; OS: overall survival; RFS: recurrence-free survival; PRS: post-recurrence survival; ALT: alanine aminotransferase; AST: aspartate aminotransferase; ALB: albumin; DFS: disease-free survival.

Acknowledgements

This work was supported by the State Key Project on Infection Diseases of China (Grant No. 2017ZX10201021-007-003), the Capital Health Research and Development of Special Fund Program (2018-1-4021), the National Natural Science Foundation of China (81672461, 81972311), CAMS Innovation Fund for Medical Sciences (CIFMS) (Grant no.2017-12M-4-002), the Non-profit Central Research Institution Fund of Chinese Academy of Medical Sciences (2019PT310026) and Sanming Project of Medicine in Shenzhen (No.SZSM202011010).

Competing Interests

The authors have declared that no competing interest exists.

References

1. Lemke J, Cammerer G, Ganser J, Scheele J, Xu P, Sander S. et al. Survival and Prognostic Factors of Colorectal Liver Metastases After Surgical and Nonsurgical Treatment. Clin Colorectal Cancer. 2016;15:e183-e92

2. Nordlinger B, Sorbye H, Glimelius B, Poston GJ, Schlag PM, Rougier P. et al. Perioperative FOLFOX4 chemotherapy and surgery versus surgery alone for resectable liver metastases from colorectal cancer (EORTC 40983): long-term results of a randomised, controlled, phase 3 trial. The Lancet Oncology. 2013;14:1208-15

3. Mitry E, Fields AL, Bleiberg H, Labianca R, Portier G, Tu D. et al. Adjuvant chemotherapy after potentially curative resection of metastases from colorectal cancer: a pooled analysis of two randomized trials. J Clin Oncol. 2008;26:4906-11

4. Leal JN, Bressan AK, Vachharajani N, Gonen M, Kingham TP, D'Angelica MI. et al. Time-to-Surgery and Survival Outcomes in Resectable Colorectal Liver Metastases: A Multi-Institutional Evaluation. J Am Coll Surg. 2016;222:766-79

5. Wei AC, Greig PD, Grant D, Taylor B, Langer B, Gallinger S. Survival after hepatic resection for colorectal metastases: a 10-year experience. Ann Surg Oncol. 2006;13:668-76

6. Viganò L, Russolillo N, Ferrero A, Langella S, Sperti E, Capussotti L. Evolution of long-term outcome of liver resection for colorectal metastases: analysis of actual 5-year survival rates over two decades. Ann Surg Oncol. 2012;19:2035-44

7. Imai K, Allard MA, Benitez CC, Vibert E, Sa Cunha A, Cherqui D. et al. Early Recurrence After Hepatectomy for Colorectal Liver Metastases: What Optimal Definition and What Predictive Factors?. Oncologist. 2016;21:887-94

8. D'Angelica M, Kornprat P, Gonen M, DeMatteo RP, Fong Y, Blumgart LH. et al. Effect on outcome of recurrence patterns after hepatectomy for colorectal metastases. Ann Surg Oncol. 2011;18:1096-103

9. de Jong MC, Pulitano C, Ribero D, Strub J, Mentha G, Schulick RD. et al. Rates and patterns of recurrence following curative intent surgery for colorectal liver metastasis: an international multi-institutional analysis of 1669 patients. Ann Surg. 2009;250:440-8

10. Takahashi S, Konishi M, Nakagohri T, Gotohda N, Saito N, Kinoshita T. Short time to recurrence after hepatic resection correlates with poor prognosis in colorectal hepatic metastasis. Jpn J Clin Oncol. 2006;36:368-75

11. Malik HZ, Gomez D, Wong V, Al-Mukthar A, Toogood GJ, Lodge JP. et al. Predictors of early disease recurrence following hepatic resection for colorectal cancer metastasis. Eur J Surg Oncol. 2007;33:1003-9

12. Viganò L, Capussotti L, Lapointe R, Barroso E, Hubert C, Giuliante F. et al. Early recurrence after liver resection for colorectal metastases: risk factors, prognosis, and treatment. A LiverMetSurvey-based study of 6,025 patients. Ann Surg Oncol. 2014;21:1276-86

13. Kaibori M, Iwamoto Y, Ishizaki M, Matsui K, Yoshioka K, Asano H. et al. Predictors and outcome of early recurrence after resection of hepatic metastases from colorectal cancer. Langenbecks Arch Surg. 2012;397:373-81

14. Takahashi S, Konishi M, Kinoshita T, Gotohda N, Kato Y, Saito N. et al. Predictors for early recurrence after hepatectomy for initially unresectable colorectal liver metastasis. J Gastrointest Surg. 2013;17:939-48

15. Liu W, Wang K, Han Y, Liang JY, Li YH, Xing BC. Nomogram predicted disease free survival for colorectal liver metastasis patients with preoperative chemotherapy followed by hepatic resection. Eur J Surg Oncol. 2019;45:2070-7

16. Liu W, Zhou JG, Sun Y, Zhang L, Xing BC. The role of neoadjuvant chemotherapy for resectable colorectal liver metastases: a systematic review and meta-analysis. Oncotarget. 2016;7:37277-87

17. Nieuwenhuizen S, Puijk RS, van den Bemd B, Aldrighetti L, Arntz M, van den Boezem PB. et al. Resectability and Ablatability Criteria for the Treatment of Liver Only Colorectal Metastases: Multidisciplinary Consensus Document from the COLLISION Trial Group. Cancers (Basel). 2020;12(7):1779

18. Chen Q, Zhao H, Wu J, Cai J, Li C, Zhao J. et al. Preoperative D-dimer and Gamma-Glutamyltranspeptidase Predict Major Complications and Survival in Colorectal Liver Metastases Patients After Resection. Transl Oncol. 2019;12:996-1004

19. Chen Q, Mao R, Zhao J, Bi X, Li Z, Huang Z. et al. Nomograms incorporating preoperative RDW level for the prediction of postoperative complications and survival in colorectal liver metastases after resection. Ann Palliat Med. 2021;10:4143-58

20. Imai K, Allard MA, Castro Benitez C, Vibert E, Sa Cunha A, Cherqui D. et al. Nomogram for prediction of prognosis in patients with initially unresectable colorectal liver metastases. Br J Surg. 2016;103:590-9

21. Okuno M, Hatano E, Seo S, Taura K, Yasuchika K, Nakajima A. et al. Indication for neoadjuvant chemotherapy in patients with colorectal liver metastases based on a nomogram that predicts disease-free survival. J Hepatobiliary Pancreat Sci. 2014;21:881-8

22. Chen Q, Mao R, Zhao J, Bi X, Li Z, Huang Z. et al. Upgraded nomograms for the prediction of complications and survival in patients with colorectal liver metastases treated with neoadjuvant chemotherapy followed by hepatic resection. Ann Transl Med. 2021;9:265

23. Rubbia-Brandt L, Giostra E, Brezault C, Roth AD, Andres A, Audard V. et al. Importance of histological tumor response assessment in predicting the outcome in patients with colorectal liver metastases treated with neo-adjuvant chemotherapy followed by liver surgery. Ann Oncol. 2007;18:299-304

24. Iasonos A, Schrag D, Raj GV, Panageas KS. How to build and interpret a nomogram for cancer prognosis. J Clin Oncol. 2008;26:1364-70

25. Życzkowski M, Prokopowicz G, Taborowski P, Nowakowski K, Rajwa P, Stelmach P. et al. Basic Parameters of Blood Count, Serum Sodium, and Creatinine as Prognostic Factors for Renal Cell Carcinoma at Five-Year Follow-Up. Med Sci Monit. 2018;24:3895-902

26. Fosså SD, Dearnaley DP, Law M, Gad J, Newling DW, Tveter K. Prognostic factors in hormone-resistant progressing cancer of the prostate. Ann Oncol. 1992;3:361-6

27. Willegger M, Posch F, Schieder S, Funovics PT, Scharrer A, Brodowicz T. et al. Serum creatinine and albumin predict sarcoma-specific survival in patients with myofibroblastic and fibroblastic sarcomas. J Orthop Res. 2017;35:2815-24

28. Panotopoulos J, Posch F, Funovics PT, Willegger M, Scharrer A, Lamm W. et al. Elevated serum creatinine and low albumin are associated with poor outcomes in patients with liposarcoma. J Orthop Res. 2016;34:533-8

29. Lafleur J, Hefler-Frischmuth K, Grimm C, Schwameis R, Gensthaler L, Reiser E. et al. Prognostic Value of Serum Creatinine Levels in Patients with Epithelial Ovarian Cancer. Anticancer Res. 2018;38:5127-30

30. Ye Z, Wang C, Guo L, Palazzo JP, Han Z, Lai Y. et al. A Routine Laboratory Data-Based Model for Predicting Recurrence After Curative Resection of Stage II Colorectal Cancer. J Natl Compr Canc Netw. 2018;16:1183-92

31. Chen J, Wang XT, Luo PH, He QJ. Effects of unidentified renal insufficiency on the safety and efficacy of chemotherapy for metastatic colorectal cancer patients: a prospective, observational study. Support Care Cancer. 2015;23:1043-8

32. Poole C, Gardiner J, Twelves C, Johnston P, Harper P, Cassidy J. et al. Effect of renal impairment on the pharmacokinetics and tolerability of capecitabine (Xeloda) in cancer patients. Cancer Chemother Pharmacol. 2002;49:225-34

33. Inaishi T, Fujita KI, Matsumoto N, Shimokata T, Maeda O, Kikumori T. et al. Correlation Between the Metabolic Conversion of a Capecitabine Metabolite, 5'-Deoxy-5-fluorocytidine, and Creatinine Clearance. In vivo. 2020;34:3539-44

34. Madi A, Fisher D, Wilson RH, Adams RA, Meade AM, Kenny SL. et al. Oxaliplatin/capecitabine vs oxaliplatin/infusional 5-FU in advanced colorectal cancer: the MRC COIN trial. Br J Cancer. 2012;107:1037-43

35. Fong Y, Fortner J, Sun RL, Brennan MF, Blumgart LH. Clinical score for predicting recurrence after hepatic resection for metastatic colorectal cancer: analysis of 1001 consecutive cases. Ann Surg. 1999;230:309-18 discussion 18-21

36. Wisneski AD, Jin C, Huang C-Y, Warren R, Hirose K, Nakakura EK. et al. Synchronous Versus Metachronous Colorectal Liver Metastasis Yields Similar Survival in Modern Era. Journal of Surgical Research. 2020;256:476-85

37. Swan PJ, Welsh FK, Chandrakumaran K, Rees M. Long-term survival following delayed presentation and resection of colorectal liver metastases. Br J Surg. 2011;98:1309-17

38. Sugihara K, Hojo K, Moriya Y, Yamasaki S, Kosuge T, Takayama T. Pattern of recurrence after hepatic resection for colorectal metastases. Br J Surg. 1993;80:1032-5

39. van der Wal GE, Gouw AS, Kamps JA, Moorlag HE, Bulthuis ML, Molema G. et al. Angiogenesis in synchronous and metachronous colorectal liver metastases: the liver as a permissive soil. Ann Surg. 2012;255:86-94

40. Gur I, Diggs BS, Wagner JA, Vaccaro GM, Lopez CD, Sheppard BC. et al. Safety and outcomes following resection of colorectal liver metastases in the era of current perioperative chemotherapy. J Gastrointest Surg. 2013;17:2133-42

41. Nuzzo G, Giuliante F, Ardito F, Vellone M, Giovannini I, Federico B. et al. Influence of surgical margin on type of recurrence after liver resection for colorectal metastases: a single-center experience. Surgery. 2008;143:384-93

42. Tsai MS, Su YH, Ho MC, Liang JT, Chen TP, Lai HS. et al. Clinicopathological features and prognosis in resectable synchronous and metachronous colorectal liver metastasis. Ann Surg Oncol. 2007;14:786-94

43. Dorcaratto D, Mazzinari G, Fernandez M, Muñoz E, Garcés-Albir M, Ortega J. et al. Impact of Postoperative Complications on Survival and Recurrence After Resection of Colorectal Liver Metastases: Systematic Review and Meta-analysis. Ann Surg. 2019;270:1018-27

44. Laurent C, Sa Cunha A, Couderc P, Rullier E, Saric J. Influence of postoperative morbidity on long-term survival following liver resection for colorectal metastases. Br J Surg. 2003;90:1131-6

45. Farid SG, Aldouri A, Morris-Stiff G, Khan AZ, Toogood GJ, Lodge JP. et al. Correlation between postoperative infective complications and long-term outcomes after hepatic resection for colorectal liver metastasis. Ann Surg. 2010;251:91-100

46. Karoui M, Penna C, Amin-Hashem M, Mitry E, Benoist S, Franc B. et al. Influence of preoperative chemotherapy on the risk of major hepatectomy for colorectal liver metastases. Ann Surg. 2006;243:1-7

47. Nakano H, Oussoultzoglou E, Rosso E, Casnedi S, Chenard-Neu MP, Dufour P. et al. Sinusoidal injury increases morbidity after major hepatectomy in patients with colorectal liver metastases receiving preoperative chemotherapy. Ann Surg. 2008;247:118-24

48. Zhao J, Sawo P, Rensen SS, Rouflart MMJ, Winstanley A, Vreuls CPH. et al. Impact of chemotherapy-associated liver injury on tumour regression grade and survival in patients with colorectal liver metastases. HPB (Oxford). 2018;20:147-54

Author contact

Corresponding address Corresponding authors: Hong Zhao MD; Tel: 87787100; E-mail: zhaohongac.cn; Zhen Huang MD; Tel: 87787100; E-mail: puragecom; Department of Hepatobiliary Surgery, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College. No. 17 Nanli, Panjiayuan, Chaoyang District, Beijing, China.


Received 2021-8-3
Accepted 2022-2-25
Published 2022-3-14


Citation styles

APA
Chen, Q., Zhang, Y., Li, X., Huang, Z., Zhao, H., Cai, J. (2022). Nomogram Incorporating Preoperative Testing Markers for the prediction of Early Recurrence for Colorectal Liver Metastases with Neoadjuvant Chemotherapy followed by Hepatectomy. Journal of Cancer, 13(6), 1758-1767. https://doi.org/10.7150/jca.65677.

ACS
Chen, Q.; Zhang, Y.; Li, X.; Huang, Z.; Zhao, H.; Cai, J. Nomogram Incorporating Preoperative Testing Markers for the prediction of Early Recurrence for Colorectal Liver Metastases with Neoadjuvant Chemotherapy followed by Hepatectomy. J. Cancer 2022, 13 (6), 1758-1767. DOI: 10.7150/jca.65677.

NLM
Chen Q, Zhang Y, Li X, Huang Z, Zhao H, Cai J. Nomogram Incorporating Preoperative Testing Markers for the prediction of Early Recurrence for Colorectal Liver Metastases with Neoadjuvant Chemotherapy followed by Hepatectomy. J Cancer 2022; 13(6):1758-1767. doi:10.7150/jca.65677. https://www.jcancer.org/v13p1758.htm

CSE
Chen Q, Zhang Y, Li X, Huang Z, Zhao H, Cai J. 2022. Nomogram Incorporating Preoperative Testing Markers for the prediction of Early Recurrence for Colorectal Liver Metastases with Neoadjuvant Chemotherapy followed by Hepatectomy. J Cancer. 13(6):1758-1767.

This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/). See http://ivyspring.com/terms for full terms and conditions.
Popup Image