J Cancer 2018; 9(17):3117-3128. doi:10.7150/jca.25339 This issue Cite

Research Paper

Preoperative CEA levels are supplementary to CA19-9 levels in predicting prognosis in patients with resectable intrahepatic cholangiocarcinoma

Chaobin He1*, Yu Zhang2*, Yunda Song1, Jun Wang1, Kaili Xing1, Xiaojun Lin1, Shengping Li1 Corresponding address

1. Department of Hepatobiliary and Pancreatic Surgery, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University Cancer Center, Guangzhou, Guangdong, 510060, P.R. China
2. State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, Guangdong, 510060, P.R. China
*These authors contributed equally to this work.

Citation:
He C, Zhang Y, Song Y, Wang J, Xing K, Lin X, Li S. Preoperative CEA levels are supplementary to CA19-9 levels in predicting prognosis in patients with resectable intrahepatic cholangiocarcinoma. J Cancer 2018; 9(17):3117-3128. doi:10.7150/jca.25339. https://www.jcancer.org/v09p3117.htm
Other styles

File import instruction

Abstract

Background: There are few diagnostic tools that can be used to determine which patient with intrahepatic cholangiocarcinoma (ICC) can benefit from surgery actually, highlighting that the need for new preoperative stratification strategies. The aim of this study was to investigate the predictive values of preoperative biomarkers in survival analyses for patients with ICC after surgical resection.

Methods: A total of 285 patients with ICC were retrospectively reviewed. Receiver operating charateristics (ROC) curves were used to evaluate the predictive effects of preoperative carbohydrate antigen 19-9 (CA19-9) with different cutoff values and carcinoembryonic antigen (CEA) in patients with ICC.

Results: Preoperative CA19-9 with a cutoff value of 200 U/ml performed better in predicting overall survival (OS) and progression free survival (PFS) in ICC patients. Patients with preoperative CA19-9 value > 200 U/ml generally had a poor surgical response. However, surgical resection could also benefit patients whose CA19-9 levels decreased postoperatively or preoperative CEA levels were negative.

Conclusions: With the cutoff value of 200U/ml, CA19-9 was a better preoperative biomarker for predicting survival for ICC patients after surgical resection. Combination of preoperative CA19-9 and CEA showed the strongest predictive power in survival analyses in these patients and should be recognized in daily clinical care.

Keywords: Intrahepatic cholangiocarcinoma, Surgical outcome, Biomarker, Prognosis

Introduction

Intrahepatic cholangiocarcinoma (ICC) represents the second most common primary hepatic tumor, accounting for 10% - 15% of all primary liver cancer [1, 2]. ICC is relatively common in Asia, with an incidence rate of 85 cases per 100.000 people a year, while the incidence rate in the US and in Western Europe is 0.5 to 3.4 cases per 100.000 people yearly [3]. Furthermore, the incidence rate of ICC has been increasing over the last few decades [4]. ICC has a more aggressive tumor biology and a worse prognosis compared with hepatocellular carcinoma [5, 6]. Surgical resection represents a potentially curative treatment; however, only 20% - 40% of patients are considered suitable for resection at the time of diagnosis [7, 8]. Moreover, the long-term survival outcome of the entire unselected patient population is unsatisfactory even after surgical resection. The 5-year overall survival rate after resection is 25% - 31% [4, 9], and the recurrence rate is as high as 50% - 60% [10, 11]. The genetic diversity and the complexity of the tumor microenvironment may contribute to the heterogeneity of ICC. The outcomes following surgery are highly variable and are difficult to predict. Surgery is not always preferable for patients because the potentially high morbidity associated with the surgery itself may outweigh the benefit. It is not clear which subset of patients actually benefit from surgical resection in terms of long-term survival, according to the standard criteria for resectable tumors [12].

Early intrahepatic recurrence is the main reason for dismal survival among ICC patients who have received surgical resection [1, 13]. Occult intrahepatic metastasis at the time of surgery may contribute to early intrahepatic recurrence [14]. Patients who are unlikely to benefit from surgical resection can potentially be identified by preoperative chemotherapy [15]. However, the implementation of surgery for patients who can benefit from resection is delayed by this time-consuming procedure. Therefore, an easily accessible parameter that can predict postoperative recurrence and long-term survival is urgently needed to optimize ICC patients for surgical resection.

The diagnosis of ICC is based on a combination of clinical, radiological, biochemical and histological approaches. There are no biomarkers, measurable in serum or in biopsy samples, currently available for the diagnosis of ICC that are sufficiently sensitive and specific. Carbohydrate antigen 19-9 (CA19-9) is the most frequently used biomarker for diagnosis in ICC patients. It was reported that preoperative CA19-9 was positively associated with tumor burden. The increase in CA19-9 may reflect early recurrence [16] or metastasis [17] in patients with ICC. However, an increase in CA19-9 may be caused by many factors, such as cholangitis [18] or obstructive jaundice [19], which are not indicative of potential recurrence or metastasis. Furthermore, it is unknown whether CA19-9 can predict survival outcomes after surgical resection in ICC patients. In addition, preoperative levels of CA19-9 have not shown enough accuracy in predicting survival for ICC patients when it was used alone in clinical practice. Thus, additional biomarkers are needed to improve efficiency in predicting long-term survival in ICC patients. Carcinoembryonic antigen (CEA), which was established as a powerful tumor marker in gastrointestinal malignancies [20, 21], has gained increasing attention as a potential tumor marker in hepatobiliary malignancies [22, 23]. Moreover, CEA levels may be independent of serum bilirubin levels [24] and an effective supplement to CA19-9 in predicting survival outcomes for ICC patients after surgical resection, especially in patients with normal preoperative CA19-9 levels. In this study, we aimed to evaluate the prognostic impact of the combination of preoperative CA19-9 and CEA in ICC patients who had received curative surgical resection.

Materials and methods

Patients

A total of consecutive 285 patients with newly pathologically proven ICC between August 1998 and September 2016 at the Department of Hepatobiliary and Pancreatic Surgery of Sun Yat-sen University Cancer Center was enrolled into this study. The whole study cohort was divided into resectable group and unresectable group. 191 patients in the resectable group underwent hepatic resection and reached the R0 standard. The rest of 94 patients in the unresectable group were not able to undergo hepatic resection due to the advanced stage disease. Ultrasound-guided fine needle aspiration biopsies were conducted to obtain tissue for histological diagnosis. The unresectable group provided a reference group to access outcomes after surgical resection. The inclusion criteria for both groups in this study were as follows: (1) liver function Child-Pugh A or B, (2) a follow-up period ≥ 6 months. The exclusion criteria were as follows: (1) liver function Child-Pugh C, (2) common diagnosis of second cancers, (3) lost to follow-up. This study was approved by the Institutional Review Board (IRB) of the Sun Yat-sen University Cancer Center. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Written informed consent was obtained from patients prior to treatment.

Data collection

All clinicopathologic and outcome data were registered in Sun Yat-sen University Cancer Center and collected for each patient. Clinical variables included age, gender, white blood cell (WBC) count, hemoglobin (HGB), platelet (PLT) count, alanine transaminase (ALT), aspartate aminotransferase (AST), alkaline phosphatase (ALP), gamma-glutamyltranspeptidase (GGT), albumin (ALB), total bilirubin (TBIL), indirect bilirubin (IBIL), C-reactive protein (CRP), CA19-9, CEA and hepatitis B surface antigen (HBsAg). Pathological variables, including tumor size, tumor number, lymph node (LN) metastasis, macrovascular invasion, microvascular invasion, lymphatic invasion, liver capsule invasion and tumor differentiation were collected and analyzed. The Tumor-Node-Metastasis (TNM) stage was determined in accordance with the new 8th American Joint Committee on Cancer (AJCC) Tumor classification of ICC [25]. Preoperative levels of biomarkers were detected within one week before surgical resection or nonsurgical treatment. Postoperative levels of CA19-9 were defined as the lowest level of CA19-9 within three months after surgical resection. The inflammation indexes, including neutrophil-to-lymphocyte ratio (NLR), platelet-to-lymphocyte ratio (PLR), albumin-to-gamma-glutamyltransferase ratio (AGR) and modified Glasgow prognostic score (mGPS), were calculated as previously described [26, 27].

Follow-up

All patients in this study were followed-up regularly once every 2 months during the first years and once every 3 months thereafter. Radiological examination, such as liver ultrasonography, computed tomogram (CT) scan, and magnetic resonance imaging (MRI), was selectively performed as needed. Hematological tests, including CA19-9, CEA test and liver function, were performed each time. Overall survival (OS) was defined as the duration from the date of the first treatment until death or last follow-up. Progression free survival (PFS) was defined as the duration from the date of operation until the date when tumor progression was diagnosed or the last follow-up. The last follow-up date was December 11, 2017. The median follow-up period was 379 days.

Statistical analysis

SPSS version 22 software (SPSS Inc., Chicago, IL, USA) was used to perform all statistical computation. The laboratory thresholds were used as cutoff values for the clinical variables. For statistical analyses, qualitative variables were analyzed by Pearson Chi-squared test or Fisher's exact test, and quantitative variables were analyzed using Mann-Whitney U test or Kruskal-Wallis test. OS and PFS were displayed using Kaplan-Meier survival curves and compared using the log-rank test. Multivariate Cox progression hazard analysis was performed for variables which were significantly associated with OS or PFS in the univariate analysis, and the corresponding 95% confidence intervals (CI) were calculated. Two-tailed P values less than 0.05 were considered statistically significant.

A time-dependent receiver operating characteristic (ROC) curve was utilized to determine the optimal cutoff values for NLR, PLR and AGR and to evaluate the discriminatory ability of the biomarkers. The calculation and comparison of integrated area under ROC curve (AUROC) were performed using R version 3.4.2 software (The R Foundation for Statistical Computing, Vienna, Austria. http://www.r-project.org).

Results

Clinical characteristics of the whole study cohort

Table 1 outlined the fundamental clinical features of patients. The clinical data include 181 male (63.5%) and 104 females (36.5%) with a median age of 55 years (range, 20-85 years). The median preoperative total bilirubin level was 11.9 umol/L and the corresponding median preoperative CA19-9 was 51.96 U/ml. The median preoperative CEA was 3.30 ng/ml. There are 166 (58.2%) patients were CA19-9 positive (≥ 35 U/ml) while a majority of patients [67.4% (192/285)] were CEA negative (< 5 ng/ml). 123 patients in the whole cohort were HBsAg-positive. The proportion of HBsAg-negative (56.8%) was a little bit higher than that of HBsAg-positive (43.2%). The optional cutoff values for NLR, PLR and AGR were 1.69, 152.35 and 0.173, respectively.

 Table 1 

Clinical characteristics of ICC patients

FeaturesN=285
Age, year, median (range)55 (20-85)
Gender (male/female)181 / 104
WBC (×109/L), median (range)7.4 (2.8-18.1)
HGB (g/L), median (range)137 (74.9-186.0)
PLT (×109/L), median (range)213 (55.0-530.0)
ALT (U/L), median (range)25.6 (8.0-322.7)
AST (U/L), median (range)27.1 (10.6-255.0)
ALP (U/L), median (range)101.8 (11.6-727.8)
GGT (U/L), median (range)81.8 (14.7-2414.3)
ALB (g/L), median (range)42.2 (26.9-52.2)
TBIL (mmol/L), median (range)11.9 (4.5-188.8)
CRP (mg/L), median (range)4.78 (0.3-250.7)
CA19-9 (U/ml), median (range)51.96 (0.6-26800.0)
CEA (ng/ml), median (range)3.3 (0.4-3803.0)
HBsAg (negative/positive)162 / 123
TNM stage (I/II/III/IV)50 / 45 /165 /25

WBC, white blood cell count; HGB, hemoglobin; PLT, platelet; ALT, alanine transaminase;

AST, aspartate aminotransferase; ALP, alkaline phosphatase; GGT, gamma-glutamyl transpeptidase;

ALB, albumin; TBIL, total bilirubin; CRP, C-reactive protein; CA19-9, carbohydrate antigen 19-9;

CEA, carcinoembryonic antigen; HBsAg, hepatitis B surface antigen; TNM, tumor-node-metastasis.

Assessment of predictive performance of CA19-9 with different cutoff values

The prognostic value of CA19-9 with different values was compared by analyzing the time-dependent ROC curves at 1-year, 2-years and 3-years follow-up (Fig. 1). As shown in Table 2, when 200 U/ml was adopted as cutoff value for CA19-9, CA19-9 was the variable with the highest AUROC value for OS analysis. What is more, ROC curves analyses revealed that CA19-9 with a cutoff value of 200 U/ml was the best predictor for PFS at 1-year and 2-years follow-up. Separately, the comparisons of AUROC implied that as a cutoff value for survival prediction, 200 U/ml was superior to 35 U/ml, which were the criteria for distinguishing normal or elevated levels of CA19-9.

 Figure 1 

Comparison of AUROC value of preoperative CA19-9 with different cutoff values in OS (A, B and C) and PFS (D, E and F) prediction. Preoperative CA19-9 with a cutoff value 200 U/ml was the best predictor for both OS and PFS for ICC patients after surgical resection.

J Cancer Image
 Table 2 

Comparison of AUROC of preoperative CA19-9 with different cutoff values in ICC patients after surgical resection

CA19-9 (U/ml)35100200300400500800100015002000
OS
1 year0.5790.6660.6690.6520.6210.6130.6070.5940.5850.585
2 years0.5510.5960.6200.6040.5750.5790.5710.5640.5590.564
3 years0.6130.6210.6430.6120.5910.5770.5980.5930.5850.581
PFS
1 year0.5860.5970.6370.6200.6000.5830.6030.5980.5870.597
2 years0.6020.5440.6100.5830.5760.5630.5730.5690.5600.575
3 years0.6370.5970.5830.5570.5560.5440.5620.5580.5500.569

OS, overall survival; PFS, progression free survival.

Comparison of pathological features of ICC patients in the resectable group separated by CA19-9 and CEA

Detailed pathological characteristics of 191 patients were summarized in Table 3. The mean tumor size was 6.22 ± 2.63 cm. The proportion of enlarged tumor (tumor size > 5cm) was higher in patients with higher levels of CA19-9 or CEA. Most of patients were without jaundice. In addition, there were no significant differences in proportions of jaundice between the two groups of normal or elevated values of biomarkers. Metastasis to LN was pathologically confirmed in 31 patients. Moreover, LN metastasis and microvascular invasion both appeared more frequently in patients with higher levels of CA19-9 or CEA. Most of patients were free of macrovascular invasion, microvascular invasion and lymphatic invasion. Separately, the proportion of liver capsule invasion and different TNM stages between two groups separated by CA19-9 with a cutoff value of 200 U/ml was significantly different while it was similar between two groups separated by CA19-9 with a cutoff value of 35 U/ml.

An elevated level of preoperative CA19-9 was an independent risk factor for both OS and PFS

In all patients within the resectable group, elevated preoperative CA19-9 serum, with a cutoff value of 200 U/ml, was significantly associated with tumor size (r = 0.249, P = 0.001), LN metastasis (r = 0.190, P = 0.008), microvascular invasion (r = 0.179, P = 0.013) and liver capsule invasion (r = 0.157, P = 0.031). A positive correlation was also observed between preoperative CA19-9 levels and TNM stage (r = 0.202, P = 0.005). Patients with lower preoperative CA19-9 levels had significantly favorable prognoses compared with patients with higher preoperative CA19-9 levels (Fig. 2, P < 0.050). For patients whose preoperative CA19-9 levels were lower than 200 U/ml, the estimated 1-, 2-, and 3-year OS rates were 86.0%, 69.3%, and 56.4%, respectively, while for patients whose preoperative CA19-9 levels were higher than 200 U/ml, the estimated 1-, 2-, and 3-year OS rates were 54.0%, 40.5%, and 20.1%, respectively. The median OS for patients with preoperative CA19-9 levels of 200 U/ml or higher was 461 days, and the median OS for patients with lower CA19-9 levels was 1295 days. Moreover, an elevated preoperative CA19-9 was also correlated with poor PFS after resection. The median PFS was significantly shorter in patients with CA19-9 levels higher than 200 U/ml compared with patients with lower levels of CA19-9 (115 days vs. 395 days, Fig. 2E, P < 0.001). Likewise, an elevated level of CEA was also associated with decreased OS (Fig. 2C, P < 0.001) and PFS (Fig. 2F, P < 0.001). Although the differences in survival rates, stratified by CA19-9 with cutoff values of both 35 U/ml and 200 U/ml were both significant, the survival of patients with CA19-9 levels less than 35 U/ml was comparable with that of patients with CA19-9 levels more than 35 U/ml when the levels of CA19-9 were less than 200 U/ml (OS, Fig. 3A, P < 0.001; PFS, Fig. 3B, P < 0.001). This result indicated that cutoff value of 35 U/ml may not be an ideal stratification for preoperative CA19-9 in survival analysis compared with the cutoff value of 200 U/ml. Since the preoperative CA19-9 cutoff value of 200 U/ml indicated the strongest predictive significance associated with survival, CA19-9 with a cutoff value of 200 U/ml was analyzed for predictive efficiency for OS and PFS. Univariate analyses indicated that both elevated CEA and CA19-9 were significantly associated with both OS and PFS (Table 4). Moreover, using multivariate Cox regression analyses, CA19-9 with a cutoff value of 200 U/ml as opposed to 35 U/ml was established as an independent predictor for both OS [hazard ratio (HR) = 2.061, 95% CI = 1.283-3.311, P = 0.003] and PFS (HR = 1.629, 95% CI = 1.050-2.528, P = 0.029). In addition to CA19-9, increasing tumor size was also a statistically significant risk predictor for poor OS (HR = 2.729, 95% CI = 1.565-4.758, P < 0.001) and PFS (HR = 2.214, 95% CI = 1.413-3.468, P = 0.001) in ICC patients after surgical resection.

 Table 3 

Relationship between biomarkers and pathological features in ICC patients after surgical resection

FeaturesCA19-9CA19-9CEA
<35U/ml≥35U/mlP value<200U/ml≥200U/mlP value<5ng/ml≥5ng/mlP value
881031415013457
Age (year)<6053600.88383301.00085280.077
≥60354358204929
Gendermale53670.54987330.61485350.870
female353654174922
Jaundiceabsent82890.158127440.788118530.440
present614146164
Tumor size (cm)<537230.005546<0.001546<0.001
≥51518087448051
Tumor numbersingle60711.00098330.72397340.091
multiple283243173723
LN matastasisabsent79810.049124360.013118420.018
present92217141615
Microvascular invasionabsent80820.042125370.020118440.077
present82116131613
Macrovascular invasionabsent84950.391133460.516127520.347
present488475
Lymphatic invasionabsent82900.228127451.000124480.111
present613145109
Liver capsule invasionabsent44460.47173170.03365250.635
present445768336932
Tumor diffrerntiationwell230.393410.469410.549
well-moderate303347164716
moderate23223783312
moderate-low414141
low294449244627
TNM stageI24260.3564460.02040100.105
II1814248248
III466373367039

Abbreviations were as in Table 1

 Figure 2 

Kaplan-Meier analyses for OS (A, B and C) and PFS (D, E and F) for ICC patients after surgical resection. Patients with elevated levels of preoperative CA19-9 with different cutoff values and CEA had a shorter both OS and PFS survival than patients with lower levels.

J Cancer Image
 Figure 3 

Kaplan-Meier analyses for OS (A) and PFS (B) based on preoperative CA19-9 with both cutoff values of 35 U/ml and 200 U/ml. A decreased survival was observed with increased preoperative CA19-9 levels higher than 200 U/ml. there were no differences in survival for patients with a cutoff value of 35 U/ml when their levels of preoperative CA19-9 were lower than 200 U/ml.

J Cancer Image

Preoperative level of CEA was a significant indicator which could stratify different survival rates for patients with preperative CA19-9 levels less than 200 U/ml

Within the resectable group, there were 50 patients who presented with a preoperative CA19-9 level of 200 U/ml or higher, while the preoperative level of CA19-9 of the remaining 141 patients was less than 200 U/ml. Even though patients with a preoperative CA19-9 level less than 200 U/ml had an improved median survival compared with patients with a higher level of preoperative CA19-9, there were also differences in survival which could be stratified by CEA in patients whose levels of preoperative CA19-9 were less than 200 U/ml (OS, Fig. 4A, P < 0.001; PFS, Fig. 4B, P < 0.001). Patients with increased preoperative CEA levels had poor OS and PFS survival times compared with patients with negative CEA levels. However, the differences in survival between CEA-negative and CEA-positive patients were not significant when they had preoperative CA19-9 levels of 200 U/ml or higher (OS, Fig. 4C, P > 0.050; PFS, Fig. 4D, P = 0.973).

 Table 4 

Univariate and multivariate cox regression analyses of OS and PFS in ICC patients after surgical resection

VariablesOSPFS
Univariate analysesMultivariate analysesUnivariate analysesMultivariate analyses
HR95%CIP valueHR95%CIP valueHR95%CIP valueHR95%CIP value
Age<60/≥ 600.8680.558-1.3510.531NI1.0310.704-1.5090.876NI
Gendermale/female1.2030.767-1.8870.422NI1.1120.751-1.6470.596NI
CA19-9 (U/ml)<35/ ≥351.6301.055-2.5190.028NS1.6941.157-2.4800.007NS
CA19-9 (U/ml)<200/ ≥2002.7041.706-4.286<0.0012.0611.283-3.3110.0032.3951.583-3.623<0.0011.6291.050-2.5280.029
CEA (ng/ml)< 5/ ≥52.6891.721-4.201<0.001NS2.3141.523-3.514<0.001NS
Tumor size (cm)< 5/ ≥53.2601.885-5.639<0.0012.7291.565-4.758<0.0012.6781.731-4.144<0.0012.2141.413-3.4680.001
Tumor numbersingle/multiple1.5921.019-2.4860.041NS1.7901.210-2.6460.004NS
LN matastasisabsent/present2.4961.456-4.279<0.0011.8521.067-3.2140.0283.3582.125-5.305<0.001NS
Microvascular invasionabsent/present1.6490.865-3.1450.129NI1.8021.095-2.9660.020NS
Macrovascular invasionabsent/present1.0710.467-2.4570.871NI1.0670.468-2.4310.877NI
Lymphatic invasionabsent/present0.4280.135-1.3550.149NI1.3620.747-2.4840.313NI
Liver capsule invasionabsent/present1.1520.748-1.7720.521NI1.5801.079-2.3130.019NS
Tumor diffrerntiationW/W-M/M/M-L/L1.0200.998-1.0430.071NI1.0040.985-1.0230.717NI
TNM stageI/II/III1.3941.162-1.672<0.001NS1.4901.262-1.760<0.0011.2391.014-1.5140.036

LN, lymph node; W, well; M, moderate; L, low; HR, hazard ratio; CI, confidence interval; NI, not include; NS, not significant; Other abbreviations as in Table 1 or Table 2.

 Figure 4 

Kaplan-Meier analyses for OS and PFS separated by preoperative levels of CEA for patients with preoperative CA 19-9 levels lower (A and B) or higher (C and D) than 200 U/ml. The differences of survival stratified by CEA were significant in patients with preoperative CA 19-9 levels lower than 200 U/ml while in a subset of patients with preoperative CA 19-9 levels higher than 200 U/ml, the survival of patients with elevated values of CEA were similar to that of patients with normal levels of CEA.

J Cancer Image

Preoperative elevated levels of CEA were independent risk factors for prognoses in patients with CA19-9 levels less than 200 U/ml or decreased postoperative levels of CA19-9

The comparisons of survival rates between categories of preoperative and postoperative CA19-9 were performed in this study. Among patients with CA19-9 levels of 200 U/ml or higher, patients with elevated postoperative CA19-9 levels had significantly decreased OS and PFS compared with patients with decreased postoperative CA19-9 levels (OS, median survival time: 235 days vs. 800 days, Fig. 5A, P < 0.001; PFS, median survival time: 66 days vs. 200 days, Fig. 5B, P = 0.017). Furthermore, the OS and PFS survival times of patients with decreased postoperative CA19-9 levels were comparable with those of patients whose preoperative CA19-9 levels were less than 200 U/ml in this study (OS, Fig. 5A, P = 0.627; PFS, Fig. 5B, P = 0.541). However, the OS rates of patients with elevated preoperative CA19-9 levels above 200 U/ml and postoperative CA19-9 levels that had either remained the same or increased were similar to those of patients in the unresectable group (Fig. 5A, P = 0.527).

Next, to identify the predictive power of preoperative CEA in patients with preoperative CA19-9 levels less than 200 U/ml or with decreased postoperative CA19-9, Kaplan-Meier survival analyses and multiple Cox regression analyses were conducted (Table 5). It was confirmed that the OS and PFS survival times of patients with negative preoperative CEA were significantly longer than those of patients with positive preoperative CEA when they had preoperative CA19-9 levels less than 200 U/ml or when they had decreased postoperative levels of CA19-9 (OS, median survival time: 1413 days vs. 461 days, Fig. 6A, P < 0.001; PFS, median survival time: 435 days vs. 142 days, Fig. 6B, P < 0.001). Moreover, univariate and multivariate Cox regression confirmed that the preoperative levels of CEA, tumor size and TNM stages were independent risk factors for both OS and PFS in patients with CA19-9 levels less than 200 U/ml or with decreased postoperative levels of CA19-9.

Assessment of predictive performance of the combination of CA19-9 and CEA

The predictive efficiencies of preoperative CEA, of CA19-9 with a cutoff value of 35 U/ml or 200 U/ml and of inflammation indexes, including NLR, PLR, AGR and mGPS, were compared in all the ICC patients after surgical resection. The AUROC values of CA19-9 with a cutoff value of 200 U/ml were higher than those of CA19-9 with a cutoff value of 35 U/ml at all times for the prediction of OS (Fig. 7A). CA19-9 with a cutoff value of 200 U/ml showed a better distinguishing power for predicting PFS within two years, while CA19-9 with a cutoff value of 35 U/ml may perform better in predicting long-term PFS (Fig. 7B). Moreover, the AUROC values were comparable between CEA and CA19-9, with a cutoff value of 200 U/ml. Most importantly, better predictive performance for both OS and PFS was achieved by the combination of CEA and CA19-9, with a cutoff value of 200 U/ml, which had the highest AUROC values at all times compared with CEA or CA19-9 alone or compared with other inflammation indexes.

 Figure 5 

OS (A) and RFS (B) of patients with elevated levels of preoperative CA19-9 were stratified by decrease or non-decrease of postoperative CA19-9. Patients with postoperative decrease of CA19-9 had better survival compared with those with non-decrease of CA19-9, which was similar to that with preoperative CA19-9 levels less than 200 U/ml while the survival of patients with non-decrease of CA19-9 postoperatively was comparable to that of patients in unresectable group.

J Cancer Image
 Table 5 

Univariate and multivariate cox regression analyses of OS and PFS in ICC patients with preoperative CA19-9 levels less than 200 U/ml or with decreased postoperative CA19-9 levels after surgical resection

VariablesOSPFS
Univariate analysesMultivariate analysesUnivariate analysesMultivariate analyses
HR95%CIP valueHR95%CIP valueHR95%CIP valueHR95%CIP value
Age<60/≥ 600.9340.583-1.4980.778NI1.0850.731-1.6090.686NI
Gendermale/female1.1490.710-1.8590.572NI1.1200.744-1.6840.588NI
CEA (ng/ml)< 5/ ≥52.4891.526-4.062<0.0012.0921.265-3.4600.0042.3541.519-3.650<0.0011.6611.054-2.6170.029
Tumor size (cm)< 5/ ≥52.8651.639-5.006<0.0012.0931.175-3.7260.0122.5271.624-3.930<0.0011.9351.221-3.0680.005
Tumor numbersingle/multiple1.8101.127-2.9050.014NS1.8761.252-2.8110.002NS
LN matastasisabsent/present2.6891.512-4.7830.001NS3.4692.153-5.588<0.0011.8151.011-3.2590.046
Microvascular invasionabsent/present1.8590.939-3.6800.075NI1.8641.116-3.1140.017NS
Macrovascular invasionabsent/present0.9210.336-2.5240.872NI1.1320.496-2.5820.769NI
Lymphatic invasionabsent/present0.4950.156-1.5730.233NI1.4590.798-2.6700.220NI
Liver capsule invasionabsent/present1.2950.813-2.0620.277NI1.6851.130-2.5120.010NS
Tumor diffrerntiationW/W-M/M/M-L/L1.0190.995-1.0440.114NI1.0020.982-1.0220.834NI
TNM stageI/II/III1.4471.186-1.765<0.0011.3691.113-1.6840.0031.5091.267-1.796<0.0011.2891.049-1.5840.016

Abbreviations were as in Table 4.

 Figure 6 

Kaplan-Meier analyses for OS (A) and PFS (B) according to preoperative levels of CEA in patients with preoperative CA19-9 levels less than 200 U/ml or decreased postoperative CA19-9 levels. Patients with preoperative CEA normal levels achieved better survival compared with those with elevated levels.

J Cancer Image
 Figure 7 

Time-dependent ROC curves analyses for predicting OS (A) and PFS (B) in ICC patients after surgical resection. Better performance for both OS and PFS were achieved by the combination of CA19-9 and CEA.

J Cancer Image

Discussion

ICC represents a rare type of cancer compared with other kinds of malignancies, despite the fact that it is the second most common primary liver malignancy [28, 29]. Studies that have focused on the diagnostic or prognostic value of biomarkers in ICC patients are scarce due to the small size of the potential study cohort [30, 31]. The survival of ICC patients after surgical resection is variable. The predictive efficiencies were relatively low for survival analyses, despite the existence of more or less standard preoperative assessment methods, such as liver function tests, clinical performance status evaluation and imaging manifestations. It was challenging and difficult to predict which patient can truly benefit from surgical resection in terms of long-term survival. It is particularly important for the identification of preoperative predictors associated with postoperative outcome because surgical resection for ICC patients is a complex procedure that is associated with potential mortality during or after surgery. Additionally, anatomic analyses are the only focus in the current resection criteria [12, 32] endorsed by the TNM staging system, which ignores some of the biological features of ICC such as its relatively high recurrence potential. Large amounts of ICC patients may experience early recurrence after surgical resection and, therefore, cannot actually benefit from surgery [33, 34]. Therefore, it is necessary to find biomarkers, especially preoperative and postoperative markers, to provide prognostic indicators which would help to select the patients who are likely to benefit from resection.

In this study, we found that, in ICC patients after surgical resection, the combination of preoperative CA19-9 and CEA was an indicator of survival outcome that showed the greatest power in predicting OS and PFS. This indicator was composed of two serum biomarkers and was easily available in the clinical workflow. CA19-9, reflecting tumor burden and activity, represents the most frequently used clinical biomarker in ICC patients [35, 36]. The proportions of patients with elevated levels of CA19-9 in the unresectable group were higher than those of patients in the resectable group in this study. In addition, we have also confirmed the stable prognostic efficiency of CA19-9 in the survival analyses and have showed an independent predictive role of CA19-9 in OS and PFS analyses in this study. These results were consistent with previous studies [37, 38]. Furthermore, some studies [39, 40] illustrated that extremely high baseline levels, which were even higher than the cutoff values for CA19-9 levels elevated beyond the normal level, were associated with poor survival in patients with pancreatic cancer. Similar to these conclusions, our study showed that CA19-9 with a cutoff value of 200 U/ml separated patients with different prognoses more accurately than a cutoff value of 35 U/ml. Patients achieved a better survival benefit form surgical resection when they showed preoperative CA19-9 levels less than 200 U/ml compared with CA19-9 levels more than 200 U/ml. The dose of 200 U/ml is, therefore, a potential threshold value of CA19-9 for stratifying surgical response in ICC patients. Studies have shown that the obstructive jaundice induced an increase in serum CA19-9 levels [41]. It was worth mentioning that most of the patients included in this study were preoperative TBIL negative. Additionally, there was no significant relationship between TBIL levels and CA19-9 levels of patients in this study (r = 0.03, P = 0.683). The influence of preoperative TBIL on the predictive accuracy of preoperative CA19-9 was minimized in our study.

Most patients with a preoperative CA19-9 level less than 200 U/ml did not have elevated CA19-9 after surgical resection. Although patients had better survival when they showed preoperative CA19-9 level less than 200 U/ml, the long-term survival rates among these patients were variable. Some patients did not achieve the same survival benefit as other patients, even though they all had preoperative CA19-9 levels less than 200 U/ml. This result highlights the biological heterogeneity of ICC and the sensitive heterogeneity of each marker for the prediction of such a progressive disease [35]. It may be that the monitoring of preoperative CA19-9 alone has limited prognostic value for ICC patients. Therefore it was necessary to include additional biomarkers to stratify patients with different survival outcomes.

CEA is a glycoprotein involved in cell adhesion processes during the fetal development of gastrointestinal tissue [42]. Currently, increasing attention has been paid to CEA as a biomarker for diagnosis and predicting prognosis in ICC patients [43, 44]. It was reported that CEA might help to identify ICC patients with an unfavorable prognosis after surgical resection [22]. We have shown that CEA was able to stratify patients with different survival even if their preoperative CA19-9 levels were less than 200 U/ml in this study. However, the differences in survival when separated by preoperative CEA levels were not significant among patients who had elevated preoperative CA19-9 levels (above 200 U/ml), revealing that the alterations of CA19-9 levels after surgical resection may have had a profound influence on the prediction of survival in ICC patients with preoperative CA19-9 levels higher than 200 U/ml.

CA19-9 has a half-life of only 14 hours, and the levels of postoperative CA19-9 will decrease or even return to normal within several weeks after surgery [45]. Although the elevated levels above 200 U/ml of preoperative CA19-9 were associated with unfavorable prognoses in ICC patients, the decline of CA19-9 levels implied an improved prognosis for ICC patients. This finding suggested that patients with elevated preoperative CA19-9 levels could also benefit from surgical resection if they achieved a decrease in CA19-9 levels after surgical resection. However, patients with elevated preoperative CA19-9 levels (above 200 U/ml) who continued to have postoperative CA19-9 levels that were stable or increased had poor survival that was comparable to that of patients in the unresectable group. This group of patients responded poorly to surgical resection and obtained no survival advantage over those in the unresectable group. Conversely, there were no significant differences in survival rates between patients with decreased postoperative CA19-9 levels and patients with preoperative CA19-9 levels less than 200 U/ml. The decrease in postoperative CA19-9 levels was the consequence of an effective reduction in tumor burden, resulting in a relative improved survival after surgical resection. In addition, our study showed that preoperative CEA could help to identify a subgroup of patients with either preoperative CA19-9 levels less than 200 U/ml or decreased postoperative CA19-9 levels who had a greater risk of recurrence and mortality, which was consistent with results of previous studies [22]. It was indicated that CEA had the potential to be used as a biomarker for diagnosis or prediction of tumor burden in CA19-9 seronegative patients who were unlikely to express CA19-9. As an effective supplement to CA19-9 in predicting survival, the elevated CEA suggested poor survival for patients who had either a postoperative decrease in CA199 levels or preoperative CA19-9 levels less than 200 U/ml. The combination of preoperative CA19-9 and CEA, showing the highest AUROC values in time-dependent ROC analyses, and it provided better performance for predicting both OS and PFS in ICC patients after surgical resection. As prognostic risk scores for ICC patients are currently not well established, the combination of preoperative CA19-9 and CEA could represent a valuable addition to the existing preoperative assessment algorithms to find an ideal and individual therapeutic approach for ICC patients.

The major limitations of the present study were its retrospective nature and the single-center design. In addition, the sample size was not sufficient in this study. Also, there was no uniform postoperative treatment for these patients, which could impact survival time and lead to deviation in the analyses. Large scale, further prospective, randomized-controlled, long-term studies are needed to confirm our results.

In conclusion, with a cutoff value of 200 U/ml, preoperative CA19-9 performs better in predicting survival for ICC patients after surgical resection. The combination of preoperative CA19-9 and CEA shows the strongest predictive value in both OS and PFS analyses in these patients and should be recognized in daily clinical care.

Acknowledgements

This work was supported by grants from the National Natural Science Foundation of China (81171890; 81672390) and the Major National Scientific Research Projects of China (No. 2013CB910304).

Competing Interests

The authors have declared that no competing interest exists.

References

1. Spolverato G, Kim Y, Alexandrescu S, Marques HP, Lamelas J, Aldrighetti L. et al. Management and Outcomes of Patients with Recurrent Intrahepatic Cholangiocarcinoma Following Previous Curative-Intent Surgical Resection. Annals of surgical oncology. 2016;23:235-43

2. Aljiffry M, Abdulelah A, Walsh M, Peltekian K, Alwayn I, Molinari M. Evidence-based approach to cholangiocarcinoma: a systematic review of the current literature. Journal of the American College of Surgeons. 2009;208:134-47

3. Bragazzi MC, Cardinale V, Carpino G, Venere R, Semeraro R, Gentile R. et al. Cholangiocarcinoma: Epidemiology and risk factors. Translational Gastrointestinal Cancer. 2011;1:21-32

4. Bridgewater J, Galle PR, Khan SA, Llovet JM, Park JW, Patel T. et al. Guidelines for the diagnosis and management of intrahepatic cholangiocarcinoma. Journal of hepatology. 2014;60:1268-89

5. Zhou XD, Tang ZY, Fan J, Zhou J, Wu ZQ, Qin LX. et al. Intrahepatic cholangiocarcinoma: report of 272 patients compared with 5,829 patients with hepatocellular carcinoma. Journal of cancer research and clinical oncology. 2009;135:1073-80

6. Konstadoulakis MM, Roayaie S, Gomatos IP, Labow D, Fiel MI, Miller CM. et al. Fifteen-year, single-center experience with the surgical management of intrahepatic cholangiocarcinoma: operative results and long-term outcome. Surgery. 2008;143:366-74

7. Weber SM, Jarnagin WR, Klimstra D, DeMatteo RP, Fong Y, Blumgart LH. Intrahepatic cholangiocarcinoma: resectability, recurrence pattern, and outcomes. Journal of the American College of Surgeons. 2001;193:384-91

8. Amini N, Ejaz A, Spolverato G, Kim Y, Herman JM, Pawlik TM. Temporal trends in liver-directed therapy of patients with intrahepatic cholangiocarcinoma in the United States: a population-based analysis. Journal of surgical oncology. 2014;110:163-70

9. Yoh T, Hatano E, Yamanaka K, Nishio T, Seo S, Taura K. et al. Is Surgical Resection Justified for Advanced Intrahepatic Cholangiocarcinoma? Liver cancer. 2016;5:280-9

10. Jiang W, Zeng ZC, Tang ZY, Fan J, Sun HC, Zhou J. et al. A prognostic scoring system based on clinical features of intrahepatic cholangiocarcinoma: the Fudan score. Annals of oncology: official journal of the European Society for Medical Oncology. 2011;22:1644-52

11. Roayaie S, Guarrera JV, Ye MQ, Thung SN, Emre S, Fishbein TM. et al. Aggressive surgical treatment of intrahepatic cholangiocarcinoma: predictors of outcomes. Journal of the American College of Surgeons. 1998;187:365-72

12. Weber SM, Ribero D, O'Reilly EM, Kokudo N, Miyazaki M, Pawlik TM. Intrahepatic cholangiocarcinoma: expert consensus statement. HPB: the official journal of the International Hepato Pancreato Biliary Association. 2015;17:669-80

13. Hyder O, Hatzaras I, Sotiropoulos GC, Paul A, Alexandrescu S, Marques H. et al. Recurrence after operative management of intrahepatic cholangiocarcinoma. Surgery. 2013;153:811-8

14. Imamura H, Matsuyama Y, Tanaka E, Ohkubo T, Hasegawa K, Miyagawa S. et al. Risk factors contributing to early and late phase intrahepatic recurrence of hepatocellular carcinoma after hepatectomy. Journal of hepatology. 2003;38:200-7

15. Buettner S, Koerkamp BG, Ejaz A, Buisman FE, Kim Y, Margonis GA. et al. The effect of preoperative chemotherapy treatment in surgically treated intrahepatic cholangiocarcinoma patients-A multi-institutional analysis. Journal of surgical oncology. 2017;115:312-8

16. Coelho R, Silva M, Rodrigues-Pinto E, Cardoso H, Lopes S, Pereira P. et al. CA 19-9 as a Marker of Survival and a Predictor of Metastization in Cholangiocarcinoma. GE Portuguese journal of gastroenterology. 2017;24:114-21

17. Wang JK, Hu HJ, Shrestha A, Ma WJ, Yang Q, Liu F. et al. Can preoperative and postoperative CA19-9 levels predict survival and early recurrence in patients with resectable hilar cholangiocarcinoma? Oncotarget. 2017;8:45335-44

18. Venkatesh PG, Navaneethan U, Shen B, McCullough AJ. Increased serum levels of carbohydrate antigen 19-9 and outcomes in primary sclerosing cholangitis patients without cholangiocarcinoma. Digestive diseases and sciences. 2013;58:850-7

19. Lin MS, Huang JX, Yu H. Elevated serum level of carbohydrate antigen 19-9 in benign biliary stricture diseases can reduce its value as a tumor marker. International journal of clinical and experimental medicine. 2014;7:744-50

20. Dolscheid-Pommerich RC, Manekeller S, Walgenbach-Brunagel G, Kalff JC, Hartmann G, Wagner BS. et al. Clinical Performance of CEA, CA19-9, CA15-3, CA125 and AFP in Gastrointestinal Cancer Using LOCI-based Assays. Anticancer research. 2017;37:353-9

21. Zhou YC, Zhao HJ, Shen LZ. Preoperative serum CEA and CA19-9 in gastric cancer-a single tertiary hospital study of 1,075 cases. Asian Pacific journal of cancer prevention: APJCP. 2015;16:2685-91

22. Loosen SH, Roderburg C, Kauertz KL, Koch A, Vucur M, Schneider AT. et al. CEA but not CA19-9 is an independent prognostic factor in patients undergoing resection of cholangiocarcinoma. Scientific reports. 2017;7:16975

23. Li Y, Li DJ, Chen J, Liu W, Li JW, Jiang P. et al. Application of Joint Detection of AFP, CA19-9, CA125 and CEA in Identification and Diagnosis of Cholangiocarcinoma. Asian Pacific journal of cancer prevention: APJCP. 2015;16:3451-5

24. Duraker N, Hot S, Polat Y, Hobek A, Gencler N, Urhan N. CEA, CA 19-9, and CA 125 in the differential diagnosis of benign and malignant pancreatic diseases with or without jaundice. Journal of surgical oncology. 2007;95:142-7

25. Amin MB ES, Greene F. AJCC Cancer Staging Manual, 8th edn. Chicago: Springer. 2017

26. He CB, Lin XJ. Inflammation scores predict the survival of patients with hepatocellular carcinoma who were treated with transarterial chemoembolization and recombinant human type-5 adenovirus H101. PloS one. 2017;12:e0174769

27. Jing CY, Fu YP, Shen HJ, Zheng SS, Lin JJ, Yi Y. et al. Albumin to gamma-glutamyltransferase ratio as a prognostic indicator in intrahepatic cholangiocarcinoma after curative resection. Oncotarget. 2017;8:13293-303

28. West J, Wood H, Logan RF, Quinn M, Aithal GP. Trends in the incidence of primary liver and biliary tract cancers in England and Wales 1971-2001. British journal of cancer. 2006;94:1751-8

29. Anderson CD, Pinson CW, Berlin J, Chari RS. Diagnosis and treatment of cholangiocarcinoma. The oncologist. 2004;9:43-57

30. Viterbo D, Gausman V, Gonda T. Diagnostic and therapeutic biomarkers in pancreaticobiliary malignancy. World journal of gastrointestinal endoscopy. 2016;8:128-42

31. Hu J, Yin B. Advances in biomarkers of biliary tract cancers. Biomedicine & pharmacotherapy = Biomedecine & pharmacotherapie. 2016;81:128-35

32. Uenishi T, Yamamoto T, Takemura S, Kubo S. Surgical treatment for intrahepatic cholangiocarcinoma. Clinical journal of gastroenterology. 2014;7:87-93

33. Zhang XF, Beal EW, Bagante F, Chakedis J, Weiss M, Popescu I. et al. Early versus late recurrence of intrahepatic cholangiocarcinoma after resection with curative intent. The British journal of surgery. 2017

34. Zhang XF, Chakedis J, Bagante F, Beal EW, Lv Y, Weiss M. et al. Implications of Intrahepatic Cholangiocarcinoma Etiology on Recurrence and Prognosis after Curative-Intent Resection: a Multi-Institutional Study. World journal of surgery. 2017

35. Yoo T, Park SJ, Han SS, Kim SH, Lee SD, Kim YK. et al. Postoperative CA19-9 Change Is a Useful Predictor of Intrahepatic Cholangiocarcinoma Survival following Liver Resection. Disease markers. 2015;2015:298985

36. Malaguarnera G, Paladina I, Giordano M, Malaguarnera M, Bertino G, Berretta M. Serum markers of intrahepatic cholangiocarcinoma. Disease markers. 2013;34:219-28

37. Shen WF, Zhong W, Xu F, Kan T, Geng L, Xie F. et al. Clinicopathological and prognostic analysis of 429 patients with intrahepatic cholangiocarcinoma. World journal of gastroenterology. 2009;15:5976-82

38. Ohtsuka M, Ito H, Kimura F, Shimizu H, Togawa A, Yoshidome H. et al. Results of surgical treatment for intrahepatic cholangiocarcinoma and clinicopathological factors influencing survival. The British journal of surgery. 2002;89:1525-31

39. Ferrone CR, Finkelstein DM, Thayer SP, Muzikansky A, Fernandez-delCastillo C, Warshaw AL. Perioperative CA19-9 levels can predict stage and survival in patients with resectable pancreatic adenocarcinoma. Journal of clinical oncology: official journal of the American Society of Clinical Oncology. 2006;24:2897-902

40. Liu L, Xu H, Wang W, Wu C, Chen Y, Yang J. et al. A preoperative serum signature of CEA+/CA125+/CA19-9 >/= 1000 U/mL indicates poor outcome to pancreatectomy for pancreatic cancer. International journal of cancer. 2015;136:2216-27

41. Humphris JL, Chang DK, Johns AL, Scarlett CJ, Pajic M, Jones MD. et al. The prognostic and predictive value of serum CA19.9 in pancreatic cancer. Annals of oncology: official journal of the European Society for Medical Oncology. 2012;23:1713-22

42. Du C, Xue W, Dou F, Peng Y, Yao Y, Zhao J. et al. Use of a combination of CEA and tumor budding to identify high-risk patients with stage II colon cancer. The International journal of biological markers. 2017;32:e267-e73

43. Brumm C, Schulze C, Charels K, Morohoshi T, Kloppel G. The significance of alpha-fetoprotein and other tumour markers in differential immunocytochemistry of primary liver tumours. Histopathology. 1989;14:503-13

44. Luo X, Yuan L, Wang Y, Ge R, Sun Y, Wei G. Survival outcomes and prognostic factors of surgical therapy for all potentially resectable intrahepatic cholangiocarcinoma: a large single-center cohort study. Journal of gastrointestinal surgery: official journal of the Society for Surgery of the Alimentary Tract. 2014;18:562-72

45. Rapellino M, Piantino P, Pecchio F, Ruffini E, Cavallo A, Scappaticci E. et al. Disappearance curves of tumor markers after radical surgery. The International journal of biological markers. 1994;9:33-7

Author contact

Corresponding address Corresponding author: Shengping Li, Department of Hepatobiliary and Pancreatic Surgery, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-sen University Cancer Center, Guangzhou, Guangdong, 510060, P.R. China. Tel +86 20 87343572; Fax +86 20 87343572; E-mail: lishporg.cn


Received 2018-2-4
Accepted 2018-6-23
Published 2018-8-6


Citation styles

APA
He, C., Zhang, Y., Song, Y., Wang, J., Xing, K., Lin, X., Li, S. (2018). Preoperative CEA levels are supplementary to CA19-9 levels in predicting prognosis in patients with resectable intrahepatic cholangiocarcinoma. Journal of Cancer, 9(17), 3117-3128. https://doi.org/10.7150/jca.25339.

ACS
He, C.; Zhang, Y.; Song, Y.; Wang, J.; Xing, K.; Lin, X.; Li, S. Preoperative CEA levels are supplementary to CA19-9 levels in predicting prognosis in patients with resectable intrahepatic cholangiocarcinoma. J. Cancer 2018, 9 (17), 3117-3128. DOI: 10.7150/jca.25339.

NLM
He C, Zhang Y, Song Y, Wang J, Xing K, Lin X, Li S. Preoperative CEA levels are supplementary to CA19-9 levels in predicting prognosis in patients with resectable intrahepatic cholangiocarcinoma. J Cancer 2018; 9(17):3117-3128. doi:10.7150/jca.25339. https://www.jcancer.org/v09p3117.htm

CSE
He C, Zhang Y, Song Y, Wang J, Xing K, Lin X, Li S. 2018. Preoperative CEA levels are supplementary to CA19-9 levels in predicting prognosis in patients with resectable intrahepatic cholangiocarcinoma. J Cancer. 9(17):3117-3128.

This is an open access article distributed under the terms of the Creative Commons Attribution (CC BY-NC) license (https://creativecommons.org/licenses/by-nc/4.0/). See http://ivyspring.com/terms for full terms and conditions.
Popup Image