PDF

J Cancer 2019; 10(3):772-778. doi:10.7150/jca.28527 This issue Cite

Research Paper

Nomogram for predicting level V lymph node metastases in papillary thyroid carcinoma with clinically lateral lymph node metastases: A large retrospective cohort study of 1037 patients from FDUSCC

Yunjun Wang^1,2, Qing Guan^1,2 , Jun Xiang^1,2

1. Department of Head and Neck Surgery, Fudan University Shanghai Cancer Center, Shanghai, 200032, China
2. Department of Oncology, Shanghai Medical Colloge, Fudan University, Shanghai, 200032, China.
Jun Xiang & Qing Guan contributed equally to this work and should be considered co-corresponding authors.

✉ Corresponding authors: Qing Guan, monique.gqcom; Jun Xiang, junxiang82com. Tel: +862164175590

Abstract

Objective: Extensive lateral neck dissection (LND), especially routine level V dissection, may potentially increase morbidity and have no significant benefit to survival in patients with papillary thyroid carcinoma (PTC). Our study aimed to determine the pattern and risk factors for level V lymph node metastasis (LNM) and to develop an assessment model based on a nomogram for predicting whether level V dissection is necessary.

Methods: A total of 1037 PTC patients with clinically suspected LNM (N1b) who underwent total thyroidectomy (TT) with central LND and unilateral LND from 2011 to 2015 were retrospectively retrieved. Univariate and multivariate analyses were performed to examine risk factors associated with level V metastasis. A nomogram for predicting level V metastasis was established and internally validated.

Results: The overall rate of level V metastasis was 21.3% (221/1037). Unilateral central LNM (CLNM), level II/III/IV metastasis, extra nodal extension (ENE) and lymph node (LN) size ≥2 cm were found to be independent predictive factors for level V metastasis (P<0.05). In the nomogram, ENE was confirmed as the largest contributor to scores, followed by LN size, unilateral CLNM and level IV/III/II metastasis. The discrimination of the prediction model was 0.738 (95% confidence interval (CI): 0.703-0.773, P<0.001).

Conclusions: The rate of level V metastasis in N1b PTC patients was apparently lower than that of other levels. With the help of our nomogram, a modified lateral neck dissection encompassing levels II-V and a strictly postoperative evaluation may be indicated when the patient has a high score.

Introduction

Papillary thyroid carcinoma (PTC) is the most common pathologic pattern of thyroid cancer and is always associated with regional cervical lymph node metastasis (LNM) at the first diagnosis. Despite the high incidence of central lymph node metastasis (CLNM), lateral lymph node metastasis (LLNM) is classified as N1b, a higher grade of the N category for PTC patients. Several retrospective studies and meta-analyses have demonstrated that LLNM is a risk factor for local recurrence and disease-free survival (DFS) [1]. It has been widely accepted that therapeutic lateral neck dissection (LND), especially modified radical neck dissection (MRND), should be performed in N1b PTC patients to improve the regional control rate [2].

The extent of MRND usually encompasses levels II-V. However, it is still controversial whether level V should be routinely dissected since the rate of LNM in this compartment is significantly low compared to that of levels II-IV [3]. In addition, although spinal accessory nerve and cervical plexus are supposed to be reserved after LND, morbidities such as shoulder dysfunction, supraclavicular numbness and neuropathic pain are frequently induced after level V dissection [4-6].

It is crucial for surgeons to preoperatively evaluate the lymph node status of level V metastases when determining the extent of LND for PTC patients. Limited studies on finding the predictors of level V metastasis have been conducted. Quantified prediction models for level V metastasis based on individual information, which are essential for clinicians to weigh treatment benefits and risks, still remain absent. A nomogram is a graphic tool for depicting the individual probability of a clinical event based on a statistical prediction model and recently has been frequently used in cancer research [7-9]. However, the application of predicting level V metastasis in PTC patients has been rarely reported. In this study, we aimed to develop a nomogram for predicting level V LNM for patients with PTC. Then, with the help of the nomogram, clinicians could effectively make a rational and evidence-based decision for the extent of LND for N1b PTC patients.

Materials and methods

Patients

A retrospective study was conducted at Fudan University Shanghai Cancer Center (FDUSCC) between January 2011 and December 2015. A total of 11577 PTC patients accepted the operation at our center, and 1037 patients who met the inclusion and exclusion criteria were enrolled in this study.

The inclusion criteria were as follows: histopathologically proven PTC and LLNM, total thyroidectomy (TT) with central lymph node dissection (CLND) and unilateral LND (at least from level II to V), and preoperative examinations (including physical examination, high-quality ultrasonography (US), computed tomography (CT) and fine needle aspiration biopsy (FNAB)) indicating suspected LLNM. The exclusion criteria were as follows: non-PTC or mixed PTC carcinomas, history of thyroid surgery, distant metastasis (DM), selective neck dissection (SND) or bilateral LND.

Surgical Strategy and histopathological examination

TT and CLND were performed first, followed by LND. The extent of CLND was the hyoid bone superiorly, the innominate vein inferiorly, and the carotid sheaths laterally, which was classified as prelaryngeal, pretracheal, ipsilateral paratracheal, or contralateral paratracheal. Thus, ipsilateral CLND encompassed the prelaryngeal, pretracheal, and ipsilateral paratracheal lymph nodes, while contralateral CLND encompassed the contralateral paratracheal lymph node. Therapeutic LND was only performed in patients with clinically suspected LLNM. LND indicates the removal of lymph nodes in the lateral neck (at least from level II to V), and MRND tends to preserve one or more nonlymphatic structures, such as the internal jugular vein, spinal accessory nerve, and sternocleidomastoid muscle on the base of LND. The neck levels were defined by the following boundaries: level II: from the base of the skull to the inferior border of the hyoid bone; level III: between the hyoid and above the inferior border of the cricoid cartilage; level IV: between the inferior border of the cricoid cartilage and the clavicle; level V: from the convergence of the sternocleidomastoid and trapezius muscles to the clavicle.

All dissected thyroid and lymph node specimens were sent to the department of pathology at FDUSCC and were examined microscopically by two or more experienced pathologists. Thyroid cancer staging and tumor size groups were categorized according to the guidelines of the American Joint Committee on Cancer (AJCC).

Clinicopathological variables assessed

The clinicopathological variables included age at diagnosis, gender, preoperative thyrotropin (TSH) level, maximal primary tumor size, multifocal/solitary lesions, primary tumor extrathyroidal extension (ETE), Hashimoto thyroiditis (HT), pathological status of CLNM and LLNM, extra nodal extension (ENE) and maximal lymph node (LN) size.

According to the 8^th edition of the AJCC staging system, patients were divided into two groups based on age: ≥55 years and < 55 years. Maximal tumor size was categorized as two groups: a small group (≤ 2 cm) and a large group (> 2 cm). ETE/ENE consisted of gross and microscopic ETE/ENE. Multifocal primary lesions were defined as two or more cancerous sites within the thyroid. Multifocal and ETE/ENE were both assessed in the final pathology.

Statistical Analysis

The Statistical Package for the Social Sciences (SPSS) for Windows, version 21.0 (SPSS Inc., Chicago, IL, United States) and the rms/pROC package in R version 3.2.2 were used in this study. All tests were two-sided, and a Pvalue less than 0.05 was considered statistically significant. Continuous variables were presented as the mean standard deviation (SD), and categorical variables were presented as the number of cases, percentages (%), and odds ratio (OR) with confidence interval (CI). Factors significant in the univariate analysis were included in the multivariate logistic regression analysis to identify independent variables. The performance of the established logistic regression model was internally validated with a bootstrapping analysis. The discriminative ability of our nomogram was assessed using the area under the receiver operating characteristic curve (AUC), also known as the concordance index. Calibration was performed to compare how well the predicted probabilities from our nomogram matched the actual probabilities of level V lymph node metastasis.

This work has been reported in line with the STROCSS criteria [10].

Results

Demographics and clinicopathological characteristics of PTC patients with LLNM

Of the 1037 PTC patients with LLNM who received unilateral LND, 326 were male and 711 were female. The mean size of the primary lesion was 1.72±1.15 cm, and 68.7% (712/1037) of the primary lesions were greater than 2 cm. Characteristics of multifocality and ETE were observed in 46.2% (479/1037) and 29.1% (302/1037) of the patients, respectively. Hashimoto's thyroiditis (HT) was found in 216 patients (20.8%) (Table 1).

Unilateral CLNM occurred in 803 patients (803/1037, 77.4%), while bilateral CLNM occurred in 281 patients (281/1037, 27.1%). The total rate of level V metastasis was 21.3% (221/1037). The incidence of metastasis in levels II, III and IV was 50.8% (527/1037), 63.9% (663/1037) and 72.8% (755/1037), respectively. The total rate of ENE was 10.3% (107/1037). The mean size of the involved LN was 1.51±0.90 cm, and 297 patients had a metastatic LN with a diameter of 2 cm or more (Table 1).

Predictive factors for level V metastasis in PTC patients with LLNM

Level V metastasis occurred much more frequently with the following factors: tumor size >2 cm, unilateral/bilateral CLNM, level II/III/IV metastasis, ENE and lymph node size ≥2 cm (P<0.05, Table 1). Then, univariate and multivariate logistic regression analyses were performed to determine the risk factors for level V metastasis (Table 2). Univariate analysis demonstrated that variables of tumor size >2 cm, unilateral/bilateral CLNM, level II/III/IV metastasis, ENE and lymph node size ≥2 cm were significantly associated with level V metastasis (P<0.05, Table 2), whereas multivariate analysis identified unilateral CLNM, level II/III/IV metastasis, ENE and lymph node size ≥2 cm as independent predictive factors for level V metastasis (P<0.05, Table 2).

Nomogram for predicting level V metastasis in PTC patients LLNM

A nomogram that incorporated the abovementioned significant factors associated with level V metastasis in PTC patients was constructed based on the logistic regression model (Figure 1a). In the nomogram, each level within variables was assigned as points on a score from 0 to 100, and ENE was confirmed as the largest contributor to scores, followed by LN size and unilateral CLNM. Level IV/III/II metastasis showed a modest impact on the model. By summing up the total score and locating it on the total point scale, a corresponding probability of level V metastasis was determined for each individual. The calibration plots presented good agreement between the bias-corrected prediction and the ideal reference line with additional 500 bootstraps in Figure 2 (mean absolute error = 0.017). Figure 3 shows that the area under curve (AUC) for the nomogram to predict level V metastasis was 0.738 (95% CI: 0.703-0.773, P<0.001). Figure 1b gives an example of how our nomogram could be used to calculate the predicted probability of level V metastasis in PTC patients.

 Figure 1 

a. Nomogram for predicting level V metastasis in PTC patients. CLNM: central lymph node metastasis; LN: lymph node; ENE: extra nodal extension. b. A 35-year-old man with PTC with CLNM, level III and level IV metastasis, 1 cm LN size, no level II metastasis and no ENE has a rate of level V metastasis of 74% (185 total points: 70 for CLNM, 58 for level IV metastasis and 57 for level III metastasis)

 Figure 2 

Calibration plots of the nomogram for predicting level V metastasis in PTC patients (internal validation set).

 Figure 3 

Discrimination plot. After 2000 bootstrap repetitions, the receiver operating characteristic analysis demonstrated that the area under curve (AUC) was 0.738 (95% CI: 0.703-0.773; P<0.001).

Discussion

In PTC patients, the incidence of regional LNM is quite common, approximately 50% to 80% based on previous reports [3]. Although PTC patients generally have a fine prognosis, LNM is associated with decreased disease-free survival (DFS) and even an increased mortality rate. The 2015 American Thyroid Association (ATA) guideline [2] recommends therapeutic LND (encompassing level II to V) for patients with biopsy-proven metastatic lateral cervical lymphadenopathy (N1b). However, some recent studies have opposed extensive LND since it has the potential for increased morbidity, such as spinal accessory nerve injury, and no significant benefit of survival compared with SND [11]. Thus, the dilemma regarding the proper extent of LND, especially whether level V should be routinely dissected, remains to be solved. Our study aimed to determine the pattern and risk factors for level V metastasis and to develop a quantified model based on a nomogram for predicting whether level V dissection is necessary.

In our study, the overall rates of LNM in levels II, III, IV and V were 50.8%, 63.9%, 72.8% and 21.3%, respectively. Level IV was the most frequently involved region, followed by levels III and II. Such high rates of metastasis for these three compartments (levels II, III and IV) were considered to be most common in N1b PTC patients, which is in-line with previous studies [3,12-14]. Kim J.S. et al. [12] demonstrated that the incidence of multilevel metastasis was 73.9% among 638 N1b patients, and this incidence was reported as being from 62.3% to 80% in several studies [15-17]. In our study, which included a large sample size, 1037 N1b PTC patients, the rate of level V metastasis was 21.3%, and this ranged from 8% to 25.3% in other series [3, 18-22]. The retrospective research of 658 N1b PTC patients by Kim J.S. also showed that level V metastasis had an unfavorable impact on overall and lateral neck recurrence [12]. Considering the high rate of LNM, multilevel involvement and worse prognosis of level V metastasis, those authors were in favor of the entire dissection of level II to V in order to achieve complete removal of metastatic lymph nodes in the lateral neck. In addition, some authors supposed that it was difficult to distinguish metastatic lymph nodes in level V metastases by physical examination, US and even CT scan and that it might lead to local recurrence by LND without compassing level V [12]. Therefore, the authors emphasized that comprehensive LND including level II-V was necessary and vital for N1b patients with PTC.

In contrast, there are conflicts in several publications in which the authors objected to routine level V dissection [23,24]. First, level V is a relatively distant anatomical compartment from levels II-IV along with the jugular chain, and thus, the incidence of LNM in level V was significantly lower than that of other compartments in our study (21.3% vs. 50.8% -72.8%, Table 1). Although imaging methods have a limited ability to identify level V metastasis preoperatively, the rate of occult metastasis in level V was only approximately 8.4% in some studies, and this rate was compared with the rate in level II-IV, which was 40% to 56% [11,15]. Second, some authors supposed that level V metastasis would increase local regional recurrence [12], but other studies concluded that there was no significant difference in the regional control rate (5-year follow-up) between MRND or SND (without level V dissection, 92% vs. 91%, P=0.89) [25]. The recurrence of level V was 3.2%-7.7% in patients with MRND, while it was 4.5% in patients with SND [11, 25]. As for postoperative morbidity, the most common injury is spinal accessory nerve injury and cervical plexus injury, which will induce shoulder dysfunction and pain (shoulder syndrome) and lower the quality of life for patients. Approximately 10% of the patients with MRND suffered from shoulder syndrome, and this proportion was only 2.7% in the patients with SND [11].

Since level V metastasis is vital for prognosis but may have an unfavorable impact on shoulder function and other aspects, it came to our attention that we should distinguish the potential predictors for level V metastasis to determine the proper extent of LND for N1b patients with PTC. We found that unilateral CLNM, level II/III/IV LNM, ENE and a max lymph node size ≥2 cm were independent risk factors. A tumor size >1 cm was once regarded as an independent predictor in Yang's research [15], but this has not been demonstrated by other studies, nor was it demonstrated by ours.

This is the first report on introducing a nomogram, a quantified risk stratification model, to predict level V metastasis in N1b PTC patients. In this nomogram, ENE was the largest contributor to scores, followed by LN size ≥2 cm, unilateral CLNM and level II/III/IV LNM. Our study demonstrated that N1b PTC patients with ENE had a much higher rate of level V metastasis compared to those without ENE (47.7% vs. 18.3%, Table 1), and this finding has never been reported in other studies [11,15,26]. We also found that the rate of level V metastasis was 24.4% in N1b PTC patients with unilateral CLNM, which was higher than that of patients with negative central lymph nodes (10.7%). Yang et al. [15] even found that an ipsilateral CLNM ratio ≥50% was an independent risk factor for level V metastasis, although other researchers did not conclude that unilateral CLNM was associated with level V metastasis. In this series, the involvement of level II, III or IV metastasis resulted in a higher tendency for level V LNM, while multilevel metastasis, especially levels II-IV metastasis simultaneously, was the only independent risk factor for level V metastasis [11]. This consequence might explain why level II/III/IV LNM had just a modest impact on this predictive model, following the other three predictors. According to previous studies, preoperative imaging examinations performed unsatisfactorily [27, 28]. Nevertheless, the AUC of this nomogram in our study was 0.738 (95% CI: 0.703-0.773, P<0.001), and an AUC value greater than 0.7 indicates good discrimination [29, 30]. Likewise, calibration plots presented good agreement between the actual probability and predicted probability of level V metastasis. Thus, utilization of our nomogram may help both clinicians and patients decide on the appropriate extent of lateral neck dissection by offering a reliable and quantified risk stratification. On the other hand, this nomogram may also contribute to the assessment of residual involved lymph node or recurrence in level V in patints with SND. Hence, we believe that our nomogram is a reliable and objective tool to help clinicians decide whether to perform level V dissection rather than making a decision based on simple and rough clinicopathological features.

Several limitations of this study exist. Since this was a nonrandomized retrospective study, it was based on histopathologically proven postoperative data which, to some extent, lacks practicability in identifying level V metastasis preoperatively. PTC subtypes, immunohistochemical staining patterns, lymphovascular invasion and other variables might be incorporated in future predictive models. Second, we failed to analyze the incidence of sublevel metastasis (which was divided by the spinal accessory nerve into Va and Vb) because sublevels of the specimens were not routinely labeled. Next, those patients who accepted selective lateral neck dissection or reoperation for level V recurrence were not enrolled in this study, which resulted in a lack of data on recurrence and effects on the strength of some outcomes. Finally, the validation of this nomogram was only conducted internally, at our single center, which might induce institutional diagnostic pattern biases. Therefore, further evaluation in external datasets is indicated. However, we believe that internal validation in this study was sufficient because of the large sample size. Despite these disadvantages, our nomogram was based on an adequate amount of N1b PTC patients with satisfactory manifestations of a good discriminative ability and internal validation.

Conclusion

This study indicated a relative lower incidence of level V LNM than levels II, IIand IV for PTC patients with clinically suspected LLNM. This finding might help to reduce the morbidity induced by routine level V dissection. A comprehensive dissection of level II-IV lymph nodes is necessary for these patients, while a more extensive lateral neck dissection should be assessed on an individual basis. Routine level V LND may be considered for N1b PTC patients with unilateral CLNM, level II/III/IV LNM, ENE and a larger lymph node size. Based on the quantified risk stratification offered by our nomogram, clinicians might perform an appropriate extent of LND for these patients individually. Level V LND and strictly postoperative evaluation may be indicated when the patient has a high nomogram score.

Abbreviations

LND: lymph node dissection; PTC: papillary thyroid carcinoma; LNM: lymph node metastasis; TT: total thyroidectomy; CLNM: central lymph node metastasis; ENE: extra nodal extension; LN: lymph node; LLNM: lateral lymph node metastasis; DFS: disease-free survival; MRND: modified radical neck dissection; FDUSCC: Fudan University Shanghai Cancer Center; CLND: central lymph node dissection; US: ultrasonography; CT: computed tomography; FNAB: fine needle aspiration biopsy; DM: distant metastasis; SND: selective neck dissection; AJCC: American Joint Committee on Cancer; TSH: thyrotropin; ETE: extrathyroidal extension; HT: Hashimoto thyroiditis; ENE: extra nodal extension; SPSS: Statistical Package for the Social Sciences; SD: standard deviation; OR: odds ratio; CI: confidence interval; AUC: area under curve; ATA: American Thyroid Association.

Acknowledgements

Sources of funding

This study was funded by the Shanghai Municipal Planning Commission of Science and Research Fund for Young Scholars (award number 20154Y0050).

Author contribution

Jun Xiang and Qing Guan conceived the study and performed the statistical analysis. Yunjun Wang participated in the study design and coordination and helped to draft the manuscript. Jun Xiang supervised the entire work. Jun Xiang & Qing Guan contributed equally to this work and should be considered co-corresponding authors. All authors read and approved the manuscript.

Competing Interests

The authors have declared that no competing interest exists.

References

1. Meer S G A, Dauwan M, Keizer B. et al. Not the number but the location of lymph nodes matters for recurrence rate and disease-free survival in patients with differentiated thyroid cancer. World J Surg. 2012;36:1262-7

2. Haugen BR, Alexander EK, Bible KC. et al. 2015 American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid. 2016;26:1-133

3. Eskander A, Merdad M, Freeman JL. et al. Pattern of spread to the lateral neck in metastatic well-differentiated thyroid cancer: a systematic review and meta-analysis. Thyroid. 2013;23:583-92

4. Laverick S, Lowe D, Brown JS. et al. The impact of neck dissection on health-related quality of life. Arch Otolaryngol Head Neck Surg. 2004;130:149-54

5. Cappiello J, Piazza C, Giudice M. et al. Shoulder disability after different selective neck dissections (levels II-IV versus levels II-V): a comparative study. Laryngoscope. 2005;115:259-63

6. Terrell JE, Welsh DE, Bradford CR. et al. Pain, quality of life, and spinal accessory nerve status after neck dissection. Laryngoscope. 2000;110:620-6

7. Guo CG, Zhao DB, Liu Q. et al. A nomogram to predict lymph node metastasis in patients with early gastric cancer. Oncotarget. 2017;8:12203-10

8. Nam RK, Kattan MW, Chin JL. et al. Prospective multi-institutional study evaluating the performance of prostate cancer risk calculators. J Clin Oncol. 2011;29:2959-64

9. Delpech Y, Bashour SI, Lousquy R. et al. Clinical nomogram to predict bone-only metastasis in patients with early breast carcinoma. Br J Cancer. 2015:113 1003-9

10. Agha RA, Borrelli MR, Vella-Baldacchino M. et al. The STROCSS Statement: Strengthening the Reporting of Cohort Studies in Surgery. Int J Surg. 2017;46:198-202

11. Kim SK, Park I, Hur N. et al. Should Level V Be Routinely Dissected in N1b Papillary Thyroid Carcinoma? Thyroid. 2017;27:253-60

12. Kim SK, Park I, Hur N. et al. Patterns, predictive factors and prognostic impact of multilevel metastasis in N1b papillary thyroid carcinoma. Br J Surg. 2017;104:857-67

13. Kupferman ME, Patterson M, Mandel SJ. et al. Patterns of lateral neck metastasis in papillary thyroid carcinoma. Arch Otolaryngol Head Neck Surg. 2004;130:857-60

14. Yanir Y, Doweck I. Regional metastases in well-differentiated thyroid carcinoma: pattern of spread. Laryngoscope. 2008;118:433-6

15. Yang J, Gong Y, Yan S. et al. Risk factors for level V lymph node metastases in solitary papillary thyroid carcinoma with clinically lateral lymph node metastases. Cancer Med. 2016;5:2161-8

16. Sivanandan R, Soo KC. Pattern of cervical lymph node metastases from papillary carcinoma of the thyroid. Br J Surg. 2011;88:1241-4

17. de Bondt RB, Nelemans PJ, Hofman PA. et al. Detection of lymph node metastases in head and neck cancer: a meta-analysis comparing US, USgFNAC, CT and MR imaging. Eur J Radiol. 2017;64:266-72

18. Noguchi S, Murakami N, Yamashita H. et al. Papillary thyroid carcinoma: modified radical neck dissection improves prognosis. Arch Surg. 1998;133:276-80

19. Shah MD, Hall FT, Eski SJ. et al. Clinical course of thyroid carcinoma after neck dissection. Laryngoscope. 2003;113:2102-7

20. Tubiana M, Schlumberger M, Rougier P. et al. Long-term results and prognostic factors in patients with differentiated thyroid carcinoma. Cancer. 1985;55:794-804

21. Lundgren CI, Hall P, Dickman PW. et al. Clinically significant prognostic factors for differentiated thyroid carcinoma: a population-based, nested case-control study. Cancer. 2006;106:524-31

22. Stack BC Jr, Ferris RL, Goldenberg D. et al. American Thyroid Association Surgical Affairs Committee 2012 American Thyroid Association consensus review and statement regarding the anatomy, terminology, and rationale for lateral neck dis- section in differentiated thyroid cancer. Thyroid. 2012;22:501-8

23. Caron NR, Tan YY, Ogilvie JB. et al. Selective modified radical neck dissection for papillary thyroid cancer: is level I, II and V dissection always necessary? World J Surg. 2006;30:833-40

24. Roh JL, Kim JM, Park CI. Lateral cervical lymph node metastases from papillary thyroid carcinoma: pattern of nodal metastases and optimal strategy for neck dissection. Ann Surg Oncol. 2008;15:1177-82

25. Xu JJ, Yu E, McMullen C. et al. Patterns of regional recurrence in papillary thyroid cancer patients with lateral neck metastases undergoing neck dissection. J Otolaryngol Head Neck Surg. 2017;46:43

26. Khafif A, Medina JE, Robbins KT. et al. Level V in therapeutic neck dissections for papillary thyroid carcinoma. Head Neck. 2013;35:605-7

27. Shim MJ, Roh JL, Gong G. et al. Preoperative detection and predictors of level V lymph node metastasis in patients with papillary thyroid carcinoma. Br J Surg. 2013;100:497-503

28. Kupferman ME, Weinstock YE, Santillan AA. et al. Predictors of level V metastasis in well-differentiated thyroid cancer. Head Neck. 2008;30:1469-74

29. Swets JA. Measuring the accuracy of diagnostic systems. Science. 1988;240:1285-93

30. Greiner M, Pfeiffer D, Smith RD. Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Prev Vet Med. 2000;45:23-41

Author contact

Corresponding authors: Qing Guan, monique.gqcom; Jun Xiang, junxiang82com. Tel: +862164175590

Received 2018-7-14
Accepted 2018-11-6
Published 2019-1-1

Citation styles

©2024 Ivyspring International Publisher. Terms of use