J Cancer 2021; 12(1):292-304. doi:10.7150/jca.51302 This issue Cite

Research Paper

Risk-predicted dual nomograms consisting of clinical and ultrasound factors for downgrading BI-RADS category 4a breast lesions - A multiple centre study

Zihan Niu1, Jia-Wei Tian2, Hai-Tao Ran3, Wei-Dong Ren4, Cai Chang5, Jian-Jun Yuan6, Chun-Song Kang7, You-Bin Deng8, Hui Wang9, Bao-Ming Luo10, Sheng-Lan Guo11, Qi Zhou12, En-Sheng Xue13, Wei-Wei Zhan14, Qing Zhou15, Jie Li16, Ping Zhou17, Chun-Quan Zhang18, Man Chen19, Ying Gu20, Jin-Feng Xu21, Wu Chen22, Yu-Hong Zhang23, Hong-Qiao Wang24, Jian-Chu Li1, Hong-Yan Wang1 Corresponding address, Yu-Xin Jiang1 Corresponding address

1. Department of Ultrasound, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100730, China.
2. Department of Ultrasound, the Second Affiliated Hospital of Harbin Medical University, Harbin 150086, China.
3. Department of Ultrasound, the Second Affiliated Hospital of Chongqing Medical University, Chongqing 400010, China; Chongqing Key Laboratory of Ultrasound Molecular Imaging, Chongqing 400010, China.
4. Department of Ultrasound, Shengjing Hospital of China Medical University, Shenyang 110004, China.
5. Department of Medical Ultrasound, Fudan University Shanghai Cancer Center & Department of Oncology, Shanghai Medical College, Fudan University, Shanghai 200032, China.
6. Department of Ultrasonography, Henan Provincial People′s Hospital, Zhengzhou 450003, China.
7. Department of Ultrasound, Shanxi Academy of Medical Science, Dayi Hospital of Shanxi Medical University, Taiyuan 030032, China.
8. Department of Medical Ultrasound, Tongji Hospital, Tongji Medical College of Huazhong University of Science and Technology, Wuhan 430030, China.
9. Department of Ultrasound, China-Japan Union Hospital of Jilin University, Changchun 130033, China.
10. Department of Ultrasound, the Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou 510120, China.
11. Department of Ultrasonography, First Affiliated Hospital of Guangxi Medical University, Nanning 530021, China.
12. Department of Medical Ultrasound, the Second Affiliated Hospital, School of Medicine, Xi'an Jiaotong University, Xi'an 710004, China.
13. Department of Ultrasound, Union Hospital of Fujian Medical University, Fujian Institute of Ultrasound Medicine, Fuzhou 350001, China.
14. Department of Ultrasound, Ruijin Hospital, Shanghai Jiaotong University, School of Medicine, Shanghai 200025, China.
15. Department of Ultrasonography, Renmin Hospital of Wuhan University, Wuhan 430060, China.
16. Department of Ultrasound, Qilu Hospital, Shandong University, Jinan 250012, China.
17. Department of Ultrasound, the Third Xiangya Hospital of Central South University, Changsha 410013, China.
18. Department of Ultrasound, the Second Affiliated Hospital of Nanchang University, Nanchang, Jiangxi, China.
19. Department of Ultrasound Medicine, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200336, China.
20. Department of Ultrasonography, the Affiliated Hospital of Guizhou Medical University, Guiyang 550004, China.
21. Department of Ultrasound, Shenzhen People's Hospital, the Second Clinical Medical College of Jinan University, Shenzhen 518020, China.
22. Department of Ultrasound, the First Hospital of Shanxi Medical University, Taiyuan 030001, China.
23. Department of Ultrasound, the Second Hospital of Dalian Medical University, Dalian 116027, China.
24. Department of Ultrasound, the Affiliated Hospital of Qingdao University, Qingdao 266003, China.

Citation:
Niu Z, Tian JW, Ran HT, Ren WD, Chang C, Yuan JJ, Kang CS, Deng YB, Wang H, Luo BM, Guo SL, Zhou Q, Xue ES, Zhan WW, Zhou Q, Li J, Zhou P, Zhang CQ, Chen M, Gu Y, Xu JF, Chen W, Zhang YH, Wang HQ, Li JC, Wang HY, Jiang YX. Risk-predicted dual nomograms consisting of clinical and ultrasound factors for downgrading BI-RADS category 4a breast lesions - A multiple centre study. J Cancer 2021; 12(1):292-304. doi:10.7150/jca.51302. https://www.jcancer.org/v12p0292.htm
Other styles

File import instruction

Abstract

Purpose: To develop and to validate a risk-predicted nomogram for downgrading Breast Imaging Reporting and Data System (BI-RADS) category 4a breast lesions.

Patients and Methods: We enrolled 680 patients with breast lesions that were diagnosed as BI-RADS category 4a by conventional ultrasound from December 2018 to June 2019. All 4a lesions were randomly divided into development and validation groups at the ratio of 3:1. In the development group consisting of 499 cases, the multiple clinical and ultrasound predicted factors were extracted, and dual-predicted nomograms were constructed by multivariable logistic regression analysis, named clinical nomogram and ultrasound nomogram, respectively. Patients were twice classified as either “high risk” or “low risk” in the two nomograms. The performance of these dual nomograms was assessed by an independent validation group of 181 cases. Receiver Operating Characteristic (ROC) curve and diagnostic value were calculated to evaluate the applicability of the new model.

Results: After multiple logistic regression analysis, the clinical nomogram included 2 predictors: age and the first-degree family members with breast cancer. The area under the curve (AUC) value for the clinical nomogram was 0.661 and 0.712 for the development and validation groups, respectively. The ultrasound nomogram included 3 independent predictors (margins, calcification and strain ratio), and the AUC value in this nomogram was 0.782 and 0.747 in the development and validation groups, respectively. In the development group of 499 patients, approximately 50.90% (254/499) of patients were twice classified “low risk”, with a malignancy rate of 1.18%. In the validation group of 181 patients, approximately 47.51% (86/181) of patients had been twice classified as “low risk”, with a malignancy rate of 1.16%.

Conclusions: A dual-predicted nomogram incorporating clinical factors and imaging characteristics is an applicable model for downgrading the low-risk lesions in BI-RADS category 4a and shows good stability and accuracy, which is useful for decreasing the rate of invasive examinations and surgery.

Keywords: Breast cancer, ultrasonography, risk factors, elastography, Nomogram

Introduction

According to global cancer statistics in 2018, breast cancer is the most common cancer in women and is the leading cause of cancer deaths (15.0% of all cancer deaths in women) [1]. Ultrasound, as a radiation-free and non-invasive method, is the preferred approach for breast examination, especially for dense breasts [2-3]. According to the American College of Radiology (ACR) Breast Imaging Reporting and Data System (BI-RADS) [4-5], breast lesions are divided into 6 categories based on different ultrasonic characteristics. Category 4 lesions have great malignant probability, varying from 2% to 95%, which are further classified into the 4a, 4b, and 4c subcategories and suggested for further examination, such as puncture biopsy or surgical treatment. The malignancy rate of type 4a lesions is only 2% to 10%, and most of them are benign, which leads to the low specificity in ultrasound. If risk prediction is performed for grade 4a lesions and low-risk lesions were conducted follow-up observation, unnecessary invasive examinations could be reduced.

The issue of how to develop a simple and effective breast cancer risk prognostic model has become the focus of breast cancer prevention. The Breast Cancer Risk Assessment Tool (BCRAT) [6], also known as the Gail Model, which is based on American Caucasian data, was proposed in 1989 by Costantito et al., through many studies and years of testing and corrections, the Gail Model has been the most widely used and is one of the standard methods for breast cancer risk assessment, especially in European and American countries. Several risk-predicted models for breast cancer have been reported with different risk factors [7-10]. Because of disparities in the various racial and ethnic groups, the application of the Gail Model has certain limitations in the Asian population [11-13]. If a 4a lesion risk-prediction model can be established based on Asian population data, the patients can be divided into "high-risk” and "low-risk” populations according to the clinical risk degree results, which will help to further improve the diagnostic accuracy and to avoid missed diagnoses.

According to BI-RADS, conventional ultrasound (US), including two-dimension (2D) and colour Doppler, could provide information for discrimination of breast lesions [14,15]. However, there are overlapping morphological features in some benign and malignant lesions. Elastography as an add-on to the conventional imaging is non-invasive and enables quantitatively assessing the tissue stiffness objectively and has been used in many diseases. In addition, elastography was added to BI-RADS in the new version in 2013, and its diagnostic value was confirmed. Based on previous studies, tissue stiffness is associated with the risk of malignancy; the harder the lesion is, the greater the probability of malignant risk is [16-18]. Ultrasound is increasingly used in clinical breast examination and has demonstrated good performance not only in breast tumour differential diagnoses but also for the potential to downgrade BI-RADS 4a lesions to reduce false-positive biopsies without increasing the risk of missing cancers [19].

Most of the previous literature used clinical factors or ultrasound signs alone to distinguish benign and malignant breast nodules. According to the study by Jieun Koh et al., BI-RADS 4a lesions were classified into “average” and “high” risk by personal or family history, and “soft” and “not soft” by elastography. Only the lesions with “average risk” and “soft stiffness” could be downgraded without further examination [19]. It was found that the missed diagnosis rate for malignant lesions was only 1.5%, and 26.7% (68/255), respectively, and benign nodules could be degraded. To evaluate clinical risk factors more comprehensively, to identify suspicious signs of traditional ultrasound and to analyse elastography quantitatively, this study established a risk factor-predicted model for downgrading 4a breast lesions, which is more suitable for the Chinese population and to reduce the rate of unnecessary examinations and surgery.

Materials and Methods

This was a multi-centre study conducted at regional medical centres in China, including 32 hospitals from 23 different provinces. All hospitals and participating radiologists completed real-name registration on the website (www.nuqcc.cn) and uploaded information after approval. To reduce the difference in diagnosis between sonographers and to improve the proficiency and consistency, before the multi-centre research, the doctors of each centre had been trained through on-site operation, demonstration and practice. All the data and images from the website were separately reviewed by three experienced radiologists in our hospitals. When there was a discrepancy, the consensus was reached after discussion. This study was registered at Chinese Clinical Trail Registry platform (http://www.chictr.org.cn) with an approval number of ChiCTR1900023916. Informed consent was obtained from all individual participants included in the study.

 Figure 1 

The flow chart of this study.

J Cancer Image

Study population

In total, 708 lesions diagnosed as 4a were selected from 3020 consecutive breast cancer patients who underwent biopsy and surgery and who were diagnosed by ultrasound from December 2018 to June 2019 in 32 hospitals.

Inclusion criteria were as follows: (1) patients who underwent breast lesion elastography examination; (2) patients with available pathological results; and (3) patients with available clinical information. The exclusion criteria included the following: (1) patients who had a history of preoperative radiotherapy, chemotherapy, or endocrine therapy; (2) patients for whom ultrasound images were not clear; and (3) patients for whom the elastography imaging was not satisfactory, such as patients with a cough who could not cooperate with elastography imaging.

A total of 680 patients were finally enrolled, including 639 with benign lesions and 41 with malignant lesions. All 4a lesions were randomly divided two groups in a ratio of 3:1 as development and validation groups, respectively. The study flow chart is shown in Figure 1. All of these patients underwent breast ultrasound examination prior to core needle biopsy or surgical pathology. The final pathologic results were considered the gold standard.

Clinical characteristics acquisition

Clinical characteristics based on the Gail Model, including personal history of breast cancer, first-degree relatives with breast cancer history (yes or no), a personal history of atypical ductal hyperplasia (ADH), and height and weight were recorded. In addition, body mass index (BMI) was calculated by the following formula: BMI = weight (kg)/height (m)2.

Ultrasonic imaging acquisition

All US examinations were performed with Resona 7 or 8 devices (Mindray Medical, Shenzhen, China) equipped with 5-14 MHz linear-array transducers. Conventional US and elastography were prospectively recorded before biopsy or surgery within two weeks by 32 sonographers with more than 3 years of experience who were blinded to the patient clinical data. For each patient, the ultrasound images of only one lesion with the highest BI-RADS categories was reserved.

The US examination included 2D and colour Doppler imaging. In 2D ultrasound, the breast tissue constitution (homogeneous background echotexture-fat, homogeneous background echotexture-fibroglandular, heterogeneous background echotexture), maximum diameters, shape (oval, round, irregular), orientation (parallel, not parallel), margin (circumscribed; one uncircumscribed feature, including indistinct, angular, microlobulated, and spiculated; more than two uncircumscribed features), echo pattern (anechoic, hyperechoic, complex cystic and solid, hypoechoic, isoechoic, heterogeneous), posterior features (no posterior features, enhancement, shadowing, combined pattern), calcification position (calcification in a mass; calcification outside of a mass; intraductal calcification), calcification morphology (micro calcification, coarse calcification), associated features (architectural distortion, duct discharges, skin changes, oedema) were collected. In Doppler ultrasound, the vascularity (absent, internal, peripheral) and the amount of blood signals (grades 0, 1, 2, 3) based on Adler's index were assessed [20]. Each lesion was categorized according to the possible malignancy of US BI-RADS (American College of Radiology, 2013) when there were one or more suspicious malignant findings, such as round or irregular shape, uncircumscribed margin, non-parallel orientation, associated features, calcification, complex echogenicity, and posterior shadowing.

The elastography model was switched after ascertaining that the lesion was in the largest diameter section and the B-mode images were optimal. To acquire reliable results, the angle of the probe was kept perpendicular to the skin and appropriate manual compression in the normal range was applied to keep the colour of the entire target stable. A wide colour spectrum of red to green to blue was displayed in elastography images, representing tissue from hard to intermediate to soft component. When the fatty tissue on the surface of the mass was blue, the elastography image was saved. Based on the colour, a Tsukuba score from 1 to 5 was assigned [21]. Scores from 1 to 5 indicated a uniform soft strain in the entire hypoechoic lesion, a mixed pattern, hard but smaller on elastogram, the same size on elastogram, and hard and larger on elastogram than in 2D images, respectively. Strain ratio was measured and recorded through drawing the region of interest location (ROI). The tumour ROI was placed entirely in the tumour, and the subcutaneous fat ROI was limited to fat not containing fibroglandular breast tissue at a similar depth to the lesion. The elastography imaging was displayed twice for every lesion and the average strain ratio was recorded.

Data and statistical analysis

Continuous variables are expressed as the mean±SD and were tested with Kruskal-Wallis rank sum test. Categorical variables are presented as frequencies and percentages and were analyzed by chi-square test. Variables showing P < 0.05 in univariate analysis were considered possible predictors and were entered in the multivariate model. Two nomograms, clinical nomogram and ultrasound nomogram, were built in the training cohort based on multivariate analysis.

Based on the significant clinical predicted factors in the development group, the patients were categorized as “clinical high risk” and “clinical low risk”. In addition, based on the ultrasonic predicted factors in the development group, the patients were divided as “ultrasound high risk” and “ultrasound low risk”.

The pathological diagnosis was used as the “gold standard”, and the area under the receiver operating characteristic (ROC) curve (AUC) was calculated after determining a cut-off value by analyzing the nomogram. The Hosmer-Lemeshow (HL) test was assessed to evaluate the calibration. TP, TN, FP, and FN represented the number of true-positive findings, true-negative findings, false-positive findings and false-negative findings, respectively.

Then, the dual nomograms built in the development group were further verified in the validation cohort. The performance of the model in terms of discrimination and diagnostic value was assessed in the validation cohort using the same methods described above.

The software SPSS Statistics (version 24.0, USA) and R software (version 3.3.0) were used for data analysis. A P value of <0.05 was considered significantly different.

Results

Clinical and pathological patient characteristics

The development group was comprised of 499 cases (473 benign lesions and 26 malignant lesions; mean age 42.13±10.85). An independent validation group included 181 cases (166 benign lesions and 15 malignant lesions; mean age 42.52±11.84). Histopathological diagnoses of the 680 breast masses were confirmed via US-guided core needle biopsy of 107 lesions and surgery of 573 lesions.

 Figure 2 

Clinical nomogram and its diagnostic value. A. Nomogram for assessing the risk degree based on clinical factors. To use this nomogram, first locate the patient's age, then draw a straight line up to the Points axis at the top to get the score associated with age. Repeat the process for the family history. Add the score of both covariates together and locate the total score on the Total points axis. Next, draw a line straight down to the “risk degree” axis at the bottom to obtain the probability. B. Receiver operating characteristic (ROC) curve in clinical risk predicted model in both develop and validation cohorts. Area under the curve (AUC) in the development set was 0.661 (Black line), and AUC in the validation set was 0.712 (Red line).

J Cancer Image

The pathology results in the development and validation cohorts are summarized in Table 1. The malignant lesion rate was 5.21% (26/499) and 8.29% (15/181) in the training and validation cohorts, respectively. The malignant lesions included ductal carcinoma in situ (DCIS), invasive ductal cancer (IDC), and invasive lobular cancer (ILC) mucinous carcinoma (MC) and invasive cancer (IC). In addition, fibroadenoma, adenosis and intraductal papilloma were the most common benign lesions.

 Table 1 

Pathological diagnosis of 450 BI-RADS category 4a breast lesions

ParameterDevelopment groupValidation group
Benign lesionsMalignant lesionsBenign lesionsMalignant lesions
Totaln=473n=26n=166n=15
Fibroadenoman=272n=97
Adenosisn=113#n=47
Intraductal papilloman=37*n=10
Abscess or mastitisn=18n=5
Hyperplasian=18n=3
Hamartoman=1n=0
Benign phyllodes tumourn=3n=1
Scarn=1n=0
Other benign diagnosesn=10n=3
Ductal carcinoma in situn=6n=7
Invasive ductal cancern=13n=3
Invasive lobular cancern=2n=4
Mucinous carcinoman=2n=0
Invasive cancern=3n=1

*One intraductal papilloma is with focal atypical ductal hyperplasia;

#One adenosis is with focal atypical ductal hyperplasia.

Clinical predicted nomogram

Base on the Gail Model and previous study, clinical characteristics, including mean age, BMI, family history of first-degree relatives with breast cancer history, age at menarche, number of births, age at first birth and ADH history, were assessed as risk factors. The results showed that only mean age and family history were significant in in both the development and validation groups (Table 2).

The variables in Table 1 were assessed in a univariate logistic regression analysis, and the variables with outcomes of P < 0.05 were entered into a multivariate logistic regression (Table 3). The results showed that age (OR =1.05, 95% CIs: 1.03 to 1.08) was an independent predictor. Many previous studies have shown that family history is strongly associated with breast cancer [22-24], even in multivariate regression, family history (OR =7.13, 95% CIs: 0.62 to 82.03) with P=0.1125, the number of first-degree relatives with history of breast cancer was an independent predictor and was forced to be incorporated into the logistic regression analysis. The formula of Linear Predictor = -4.78455 +0.04167*Age +1.96384*(family history=1). A model incorporating these two independent predictive factors was built and is shown as a nomogram (Figure 2A). To use the nomogram, first, the subject's age and family history can be located on the relevant axis. Next, a straight line is drawn upwards, to the point of the axis on the top, to acquire the points received based on covariates, respectively. Total points are calculated by adding all the points obtained from every covariate. The final sum is located on the total points axis, and a straight line was drawn downwards from there to obtain the probability of risk degree. Through the nomogram, the cut-off of risk degree was 0.0593. These patients were regarded as “clinical high risk” with risk degree more than or equal to 0.0593, and as “clinical low risk” with risk degree less than 0.0593.

In internal validation, the ROC showed the resulting model with an AUC of 0.661. The Hosmer-Lemeshow test was not significant (P = 0.694), suggesting a good fit of the model. In the independent validation cohort, the clinical model displayed moderate discrimination with an AUC of 0.712 (Figure 2B). Moreover, the Hosmer-Lemeshow test (P =0.358 ) was not significant. The diagnostic value of the clinical nomogram in the development and validation groups is shown in Table 4. In the development group, the sensitivity, specificity and accuracy were 0.5385, 0.7526 and 0.7415, respectively. With the same cut-off as the development group, the sensitivity, specificity and accuracy in the validation group were 0.5333, 0.7831 and 0.7624, respectively.

Ultrasound-predicted nomogram

According to BI-RADS, the ultrasound characteristics and their P values are shown in Table 5. The results showed that margin, shape, calcification morphology, position and elastography index were significantly different in the development group. In addition to the above parameters, there were other parameters, such as echo pattern and structural distortion, which had certain significant differences in the validation group.

The variables in Table 5 were assessed in a univariate logistic regression analysis, and the variables with outcomes of P < 0.05 were entered into a multivariate logistic regression (forward stepwise logistic regression) (Table 3), where variables with P < 0.05 were considered possible predictors. Margin, calcification, and strain ratio were identified as independent predictors of patient classification as “high risk” or “low risk”. The formula of Linear Predictor = -5.46644 +1.44638(Margin=1) +2.50926*(Margin=2) +0.26413*(Calcification=1) +1.68924*(Calcification=2) +0.32118*Strain ratio. A model was built and is shown as a nomogram (Figure 3A).

The probability of ultrasound risk degree was obtained from the nomogram in the same method as above. The total points are calculated by adding all the points obtained from margin, calcification morphology, and strain ratio and the cut-off of risk degree was 0.0486. Patients were regarded “ultrasound high risk” with risk degree more than or equal to 0.0486, and “ultrasound low risk” with risk degree less than 0.0486.

In the development and validation cohorts, the discrimination of ultrasound risk nomogram was moderate with AUC of 0.782 (P = 0.905) and 0.747 (P = 0.359), respectively (Figure 3B). In addition, the Hosmer-Lemeshow test was not significant.

The dual nomogram diagnostic process in the development group

For the dual nomogram established in this research, the internal validation was performed in the development group. First, the patients were divided into high-risk 26.25% (131/499) and low-risk 73.75% (368/499) groups based on clinical nomogram. Then, through the ultrasound nomogram, the patient was given a second risk-degree classification (Figures 4 & 5).

 Table 2 

Clinical characteristics of category BI-RADS 4a breast lesions and P values

CharacteristicsDevelopment groupValidation group
Benign (n=473)Malignant (n=26)P-valueBenign (n=166)Malignant (n=15)P-value
Mean age41.85 ± 10.7947.27 ± 10.92<0.008*41.72 ± 11.3751.47 ± 13.630.007*
BMI22.43 ± 2.7622.68 ± 2.780.51122.73 ± 2.7623.58 ± 2.310.149
Family history0.028*0.031*
No471 (99.58%)25 (96.15%)165 (99.40%)14 (93.33%)
Yes2 (0.42%)1 (3.85%)1 (0.60%)1 (6.67%)
Age at menarche0.6060.349
7-1131 (6.57%)3 (11.54%)11 (6.63%)2 (13.33%)
12-13259 (54.87%)13 (50.00%)95 (57.23%)10 (66.67%)
≥14183 (38.69%)10 (38.46%)60 (36.14%)3 (20.00%)
Number of births0.3960.058
074 (15.65%)3 (11.54%)25 (15.06%)0 (0.00%)
1250 (52.85%)17 (65.38%)86 (51.81%)8 (53.33%)
2129 (27.27%)4 (15.38%)47 (28.31%)4 (26.67%)
≥320 (4.23%)2 (7.69%)8 (4.82%)3 (20.00%)
Age at first birth0.3800.204
No birth74 (15.64%)3 (11.54%)25 (15.06%)0 (0.00%)
<25188 (39.75%)10 (38.46%)70 (42.17%)10 (66.67%)
≥25, <30178 (37.63%)13 (50.00%)62 (37.35%)4 (26.67%)
≥3033 (6.98%)0 (0.00%)9 (5.42%)1 (6.67%)
History of ADH0.6840.763
No470 (99.37%)26 (100.00%)165 (99.40%)15 (100.00%)
Yes3 (0.63%)0 (0.00%)1 (0.60%)0 (0.00%)

*P value was calculated by comparing benign and malignant lesions in pathology and indicates significant difference;

BI-RADS, Breast Imaging Reporting and Data System; BMI, Body Mass Index.

 Figure 3 

Ultrasound nomogram and its diagnostic value. A. Nomogram for assessing the risk degree based on ultrasonic-predicted factors. Draw a straight line up to the Points axis at the top to get the score associated with margin, calcification and strain ratio, and add the score of both covariates together and locate the total score on the Total points axis. Next, draw a line straight down to the “risk degree” axis at the bottom to obtain the probability. B. Receiver operating characteristic (ROC) curve for ultrasonic risk predicted model in both the development and validation cohorts. ROC curve of diagnostic performance for malignant lesions with AUC of 0.782 in the development group and 0.747 in the validation cohort.

J Cancer Image
 Table 3 

Univariate analysis and multivariate logistic regression analysis in the development cohort

VariablesUnivariate logistic analysisMultivariate logistic analysis
ORs (95% CIs)PORs (95% CI)P
Age1.05(1.03, 1.08)0.0001*1.042 (1.00, 1.08)0.0180*
BMI1.06 (0.95, 1.18)0.2804
Age at menarche
7-111.0 (Reference)
12-130.55 (0.20, 1.51)0.2439
≥140.45 (0.15, 1.33)0.1495
Number of births
01.0 (Reference)
12.42 (0.72, 8.20)0.15451.04 (0.27, 3.98)0.9577
21.48 (0.39, 5.73)0.56590.45 (0.09, 2.30)0.3379
≥36.05 (1.36, 26.93)0.0182*0.92 (0.11, 7.71)0.9374
Family history
01.0 (Reference)
110.87 (1.76, 66.98)0.0101*7.13 (0.62, 82.03)0.1125
Tissue composition
Heterogeneous background echotexture1.0 (Reference)1.0
Homogeneous background echotexture-fibroglandular4.09 (0.54, 30.92)0.1723
Homogeneous background echotexture-fat8.49 (0.96, 74.99)0.0543
Posterior features
No posterior features change1.0 (Reference)
Enhancement0.85 (0.24, 2.97)0.7995
Combined pattern2.75 (0.58, 13.05)0.2018
Shadowing1.31 (0.37, 4.64)0.6711
Echo pattern
Heterogeneous1.0 (Reference)
Isoechoic0.00 (0.00, Inf)0.9875
Hypoechoic0.41 (0.09, 1.92)0.2596
HyperechoicNANA
Complex cystic and solid0.71 (0.11, 4.65)0.7173
Anechoic0.00 (0.00, Inf)0.9969
Distance to the nipple1.00 (0.79, 1.27)0.9836
Largest diameter1.25 (0.86, 1.82)0.2489
Location
Right1.0 (Reference)
Left1.07 (0.48, 2.35)0.8749
Orientation
Not-parallel1.0 (Reference)
Parallel0.44 (0.19, 1.04)0.0610
Margin
Clear1.0 (Reference)
One feature3.71 (1.73, 7.98)0.00084.25 (1.56, 11.54)0.0046*
Two or more features11.89 (4.48, 31.56)<0.0001*12.30 (3.38, 44.75)0.0001*
Calcification
No calcification1.0 (Reference)
Coarse calcification0.86 (0.25, 2.92)0.80990.98 (0.22, 4.40)0.9766
Micro-calcification3.90 (1.90, 7.98)0.0002*4.67 (1.94, 11.26)0.0006*
Shape
Oval1.0 (Reference)
Round0.00 (0.00, Inf)0.9860NA0.9902
Irregular8.28 (1.99, 34.38)0.0036*4.80 (0.82, 27.91)0.0809
Blood flow signal
01.0 (Reference)
10.83 (0.17, 3.97)0.8127
21.21 (0.26, 5.54)0.8099
32.26 (0.50,10.19)0.2876
Strain ratio1.32 (1.03, 1.69)0.0282*1.38 (1.04, 1.83)0.0264*

*P value indicates significant difference.

ORs: odds ratios; CIs: confidence intervals; NA: not applicable.

Of 131 lesions in clinical high-risk women, 37.40% (49/131) of lesions showed high-ultrasonic risk and 62.60% (82/131) showed low risk by ultrasound. The malignancy rate of lesions with dual high risk was 20.41% (10/49). Of 368 lesions in clinical low-risk women, 30.98% (114/368) showed high-ultrasonic risk and 69.02% (254/368) showed low risk by ultrasound. Of 254 lesions with both clinical and ultrasonic low risk, only 3 lesions were malignant and diagnosed as IDC pathologically, of which the malignancy rate was 1.18% and far below those with dual high-risk lesions. When either the clinical or ultrasonic risk was high, the malignancy rate was 4.88% and 7.89%, respectively (Figure 6).

 Figure 4 

Images from a 54-year-old woman with a BI-RADS category 4a lesion in the left breast and the pathology was invasive ductal carcinoma. (A) Lesion size was 3.2*1.4 cm on B-mode imaging in the longest axis of the mass. The lesion showed unclear boundary. (B) The third measurement of the lesion from a view orthogonal to the first image. (C) Grade 1 in Colour Doppler. (D) The elastic ratio was 4.13 (blue represents soft and red represents hard). (E) In the clinical nomogram, the patient was high risk with 0.0735. (F) The lesion was classified as high risk in the ultrasound nomogram with a risk rate of 0.0634.

J Cancer Image
 Table 4 

The diagnostic performance of clinical and ultrasound nomograms in the development and validation groups

NomogramGroupAUCCIsTPTNFPFNSeSpAcLR+LR-PPVNPV
Clinical nomogramDevelopment group0.66120.5464-0.776114117123560.53850.75260.74152.17690.61320.10690.9674
Validation group0.71180.5766-0.847183671300.53330.78310.76242.45920.59590.18180.9489
Ultrasound nomogramDevelopment group0.78240.6987-0.86611914473290.73080.69560.69742.40040.38710.11660.9792
Validation group0.74660.5848-0.9084114841180.71080.73330.71272.53610.37510.18640.9672

Abbreviations: AUC, area under the curve; CIs, confidence intervals; TP, true positive; TN, true negative; FP, false positive; FN, false negative; Se, sensitivity; Sp, specificity; Ac, accuracy; LR+, positive likelihood ratio; LR-, negative likelihood ratio; PPV, positive predictive values; NPV, negative predictive values.

 Table 5 

Ultrasound features of category BI-RADS 4a breast lesions and P values

Ultrasound featuresTraining groupValidation group
Benign (n=473)Malignant (n=26)P-valueBenign (n=166)Malignant (n=15)P-value
Distance to the nipple (cm)2.44±1.542.48±1.610.9602.37±1.533.37 ± 1.690.027
Largest diameter (cm)1.47±0.891.68±0.760.0521.50±0.871.59±0.690.225
Location0.8751.000
Right244 (51.69%)13 (50.00%)75 (45.18%)7 (46.67%)
Left229 (48.41%)13 (50.00%)91 (54.82%)8 (53.33%)
Tissue composition0.0940.311
Heterogeneous background echotexture73 (15.43%)1 (3.85%)31 (18.67%)4 (26.67%)
Homogeneous background Echotexture-fibroglandular357 (75.48%)20 (76.92%)126 (75.90%)9 (60.00%)
Homogeneous background Echotexture-fat43 (9.09%)5 (19.23%)9 (5.42%)2 (13.33%)
Shape<0.014*NA
Irregular6 (1.27%)3 (11.54%)0 (0.00%)0 (0.00%)
Oval463 (97.89%)23 (88.46%)166 (100%)15 (100.00%)
Round4 (0.85%)0 (0.00%)0 (0.00%)0 (0.00%)
Orientation0.4540.067
Not-parallel37 (7.82%)3 (11.54%)16 (9.64%)4 (26.67%)
Parallel436 (92.18%)23 (88.46%)150 (90.36%)11 (73.33%)
Margin<0.001*0.005*
Clear277 (58.56%)6 (23.08%)106 (63.86%)4 (26.67%)
One feature174 (36.79%)14 (53.85%)53 (31.93%)8 (53.33%)
Two or more features22 (4.65%)6 (23.08%)7 (4.22%)3 (20.00%)
Posterior features0.5620.116
No posterior features change347 (73.36%)18 (69.23%)117 (70.48%)7 (46.67%)
Enhancement68 (14.38%)3 (11.54%)29 (17.47%)5 (33.33%)
Combined pattern14 (2.96%)2 (7.69%)7 (4.22%)0 (0.00%)
Shadowing44 (9.30%)3 (11.54%)13 (7.83%)3 (20.00%)
Echo pattern0.5590.006*
Heterogeneous16 (3.38%)2 (7.69%)6 (3.61%)1 (6.67%)
Isoechoic16 (3.38%)0 (0.00%)6 (3.61%)0 (0.00%)
Hypoechoic407 (86.05%)21 (80.77%)147 (88.55%)11 (73.33%)
Hyperechoic0 (0%)0 (0.00%)0 (0.00%)1 (6.67%)
Complex cystic and solid33 (6.98%)3 (11.54%)7 (4.22%)2 (13.33%)
Anechoic1 (0.21%)0 (0.00%)0 (0.00%)0 (0.00%)
Calcification position0.028*0.354
No calcification374 (79.07%)15 (57.69%)128 (77.11%)10 (66.67%)
In a mass96 (20.30%)11 (42.31%)38 (22.89%)5 (33.33%)
Outside of a mass3 (0.63%)0 (0%)00
Calcification morphology0.003*0.229
0374 (79.07%)15 (57.69%)128 (77.11%)10 (66.67%)
Coarse calcification51 (10.78%)2 (7.69%)19 (11.45%)1 (6.67%)
Micro-calcification48 (10.15%)9 (34.62%)19 (11.45%)4 (26.67%)
Structural distortion1.000<0.001*
No471 (99.58%)26 (100.00%)166 (100.00%)12 (80.00%)
Yes2 (0.42%)0 (0.00%)0 (0.00%)3 (20.00%)
Duct discharges0.7160.369
No431 (91.12%)25 (96.15%)148 (89.16%)15 (100.00%)
Yes42 (8.88%)1 (3.85%)18 (10.84%)0 (0.00%)
Skin Change1.0001.000
Yes5 (1.06%)0 (0.00%)2 (1.20%)0 (0.00%)
No468 (98.94%)26 (100.00%)164 (98.80%)15 (100.00%)
Edema1.0001.000
No472 (99.79%)26 (100.00%)165 (99.40%)15 (100.00%)
Yes1 (0.21%)0 (0.00%)1 (0.60%)0 (0.00%)
Blood flow sign0.0970.052
328 (5.92%)2 (7.69%)13 (7.83%)0 (0.00%)
2110 (23.26%)9 (34.62%)44 (26.51%)8 (53.33%)
1167 (35.31%)3 (11.54%)56 (33.73%)6 (40.00%)
0168 (35.52%)12 (46.15%)53 (31.93%)1 (6.67%)
Elastography score<0.001*0.141
135 (7.40%)0 (0.00%)18 (10.84%)2 (13.33%)
2135 (28.54%)1 (3.85%)45 (27.11%)1 (6.67%)
3206 (43.55%)8 (30.77%)73 (43.98%)6 (40.00%)
486 (18.18%)16 (61.54%)27 (16.27%)6 (40.00%)
511 (2.33%)1 (3.85%)3 (1.81%)0 (0.00%)
Strain ratio3.47±1.384.08±1.200.010*3.37 ± 1.343.46 ± 1.650.352

*P value was calculated by comparing benign and malignant lesions in pathology and indicates significant difference.

 Figure 5 

Image of a fibroadenoma in a 25-year-old woman with a BI-RADS category 4a lesion in the right breast. (A) Lesion size was 0.8*0.6 cm on B-mode imaging in the longest axis of the mass. The lesion showed indistinct margin. (B) The third measurement of the lesion from a view orthogonal to the first image is 0.7 cm. (C) Grade 0 in Colour Doppler. (D) The elastic ratio was 2.89 (blue represents soft and red represents hard). (E) In the clinical nomogram, the patient was classified as low risk with 0.0231. (F) The lesion was classified as low risk in the ultrasound nomogram with risk degree of 0.0434.

J Cancer Image

The dual nomogram diagnostic process in the validation group

In the validation group, 181 lesions, 27.62% (50/181) showed high clinical risk and 72.38% (131/181) low clinical risk (Figure 7). The malignancy rate of lesions in women with both ultrasound and clinical low risks was 1.16% (1/86) and was significantly lower than the respective 42.86% (6/14) malignancy rate of lesions in women with both high clinical and ultrasound risks.

When one of the two nomograms was high risk, the malignancy rate could be 8.33% (3/36) and 11.11% (5/45), respectively. Through this model, approximately 47.51% (86/181) of lesions with dual low-risk lesions could be downgraded with a missed diagnosis rate of only 1.16%.

Discussion

Breast cancer has become a disease of global concern. Category 4a lesions have a malignancy rate of 2% to 10%, with low specificity in ultrasound diagnosis, which leads to unnecessary invasive examination of benign diseases. To establish an efficient model for downgrading 4a lesions, this study constructed dual nomograms based on a Chinese sample, and the patients were twice classified as either “high risk” or “low risk” in clinical and ultrasound nomograms. The diagnostic performance of dual nomograms was validated in another group. The results showed that whether internal or external validation, the model constructed in this study can effectively discriminate 4a lesions.

There are currently multiple breast cancer risk assessment models, and the existing risk prediction models have similar, moderate predictive accuracy overall [25]. The Gail Model is the most widely used and is one of the standard methods for breast cancer risk assessment. Moreover, its predicted value has been assessed in previous studies with different results. According to Gao et al., the model containing only age at menarche, age at first birth and number of first-degree relatives with breast cancer could provide a more convenient way to predict the risk of invasive breast cancer in Southeast Asian women [26]. Sa-Nguanraksa D conducted a study to evaluate whether the Gail Model can calculate the risk of breast cancer in Thai women and found age, parity, age at first live birth, and history of atypical ductal hyperplasia (ADH) were significant risk factors for breast cancer [27]. In our study, the patient's age and family history are two of the most significant variables for patients with 4a lesions, which is similar to previous studies. Other factors, such as ADH and age at first birth, were not associated with breast cancer risk, which may be due to the differences in the sample population enrolled and sample size restrictions.

 Figure 6 

The diagnostic process of dual nomogram in the development group. Abbreviations: HR, high risk; LR, low risk.

J Cancer Image
 Figure 7 

The diagnostic process of dual nomogram in the validation group. Abbreviations: HR, high risk; LR, low risk.

J Cancer Image

According to BI-RADS, 4a category lesions have certain malignancy signs, but benign and malignant are not easy to distinguish. The ultrasound findings of some benign and malignant lesions in this study were overlapping. In the development group, 88.46% of malignant nodules grew in parallel, and 23.08% of lesions had clear margins. Furthermore, benign lesions could also show malignant signs; for example, 41.44% had unclear margins and 7.82% were non-parallel. For 4a lesions, selecting the riskiest malignant signs is helpful to distinguish the lesions and to improve the diagnostic value. This study found that margins and calcified morphology were significantly different. According to logistic regression analysis, the OR value of coarse calcification and micro-calcification was 0.98 (P=0.9766) and 4.67 (P=0.0006), respectively. Micro-calcification was significantly associated with malignant lesions, which is the same as previous studies [28]. In addition, the margins in this study were highly significant for the diagnosis of 4a lesions. Unclear margins include indistinct, angular, microlobulated and spiculated, and when more than two cases are combined, the rate of malignancy risk is higher.

In the new version of BI-RADS guidelines, elastography as a new predictor has been added to assess breast lesions and has been reported with a high diagnostic value in the differentiation of breast lesions in many studies. In a meta-analysis of 2087 lesions by 9 studies, Sadigh et al. summarized the accuracy of elastography for differentiation of malignant and benign breast abnormalities and found the pooled sensitivity of 88% and specificity of 83% when using strain ratio, and the pooled sensitivity of 98% and specificity of 72% when using length ratio [29].

Han et al. compared different elastic methods, including strain elastography (SE), acoustic radiation force impulse-inducing Virtual Touch Imaging (VTI), and Virtual Touch Imaging Quantification (VTIQ), in downgrading US BI-RADS category 4a lesions. The authors found that 50.8% to 85.8% of lesions were downgraded with a malignancy rate range from 0% to 50% when using different combinations of elastography methods, which showed the combination of different elastic methods has the potential to downgrade BI-RADS 4a lesions with excellent performance [30]. This study added shear elastography on the basis of conventional ultrasound and combined clinical characteristics, and approximately 50.90% in the development group and 47.51% in the validation group 4a lesions with both low risks could be downgraded with the false negative rate of only 1.18% or 1.16%, respectively, which increased diagnostic value for the identification of benign and malignant lesions and showed excellent stability. The malignancy rate of BI-RADS 4a lesions with dual “low risks” is in line with the proportion of BI-RADS 3 lesions (<2%) and could be followed up by this dual nomogram, which is conducive to reducing unnecessary invasive biopsies and saving clinical resources.

There are some limitations in this study. First, only patients with pathology results were enrolled in this study, and some patients in follow-up were not included, which may have led to selection bias and resulted in the underestimation of the NPV and the overestimation of the PPV. Second, the predicted value of breast cancer family history (only 3 in the benign group and 2 in the malignant group with a positive breast cancer family history) may be limited because of the moderate sample size of the study. We used the forced inclusion logistic regression and incorporated “family history” as one risk factor to reduce the deviation of modal.

In this study, we extracted meaningful factors from a Chinese multi-centre sample and established a dual-risk predicted model by clinical and ultrasound factors. The patients who had both low risks could be downgraded and could be followed up with a low malignancy rate, which was less than 2%. In conclusion, this dual nomogram, taking clinical risk factors and ultrasound characteristics together, showed high predictive value for downgrading US BI-RADS 4a lesions in both the development and validation groups, which is useful to reduce invasive examinations and surgery, and can be used as a screening tool for risk stratification among the 4a lesions.

Acknowledgements

Funding

This work is supported by the Natural Science Foundation of Beijing (7202156), Foundation of ihecc2018C0032-2 and Peking Union Medical College Reform in Education Project (10023201900113).

Ethical approval

This study was approved by the ethics committee of the principal investigator's hospital (Peking Union Medical College Hospital) and is registered at ClinicalTrials.gov (ChiCTR1900023916). Informed consent was obtained from all individual participants included in the study.

Competing Interests

The authors have declared that no competing interest exists.

References

1. Bray F, Ferlay J, Soerjomataram I. et al. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortalityworldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2018;68(6):394-424 doi: 10.3322/caac.21492. Epub 2018 Sep 12

2. Elizabeth H Dibble, Tisha M Singer, Nneka Jimoh. et al. Dense breast ultrasound screening after digital mammography versus after digital breast tomosynthesis. R Am J Roentgenol. 2019;213:1397-1402

3. Berg WA. Current Status of Supplemental Screening in Dense Breasts. J Clin Oncol. 2016;34:1840-1843

4. American College of Radiology, BI-RADS Committee. ACR BI-RADS breast imaging and reporting data system: breast imaging atlas. Reston, VA, American College of Radiology. 2003

5. Mendelson EB, Bohm-Velez M, Berg WA. et al. ACR BI-RADS Ultrasound. In: ACR BI-RADS Atlas, Breast Imaging Reporting and Data System. Reston, Va:.American College of Radiology. 2013

6. Breast Cancer Risk Assessment Tool. National Cancer Institute. 2011. https://www.cancer.gov/bcrisktool/

7. Cintolo-Gonzalez J A, Braun D, Blackford A L. et al. Breast cancer risk models: a comprehensive overview of existing models, validation, and clinical applications[J]. Breast Cancer Research and Treatment. 2017;164(2):263-284

8. Howell A, Anderson AS, Clarke RB. et al. Risk determination and prevention of breast cancer. Breast Cancer Res. 2014;16:446

9. Antoniou AC, Cunningham AP, Peto J. et al. The BOADICEA model of genetic susceptibility to breast and ovarian cancers: updates and extensions. Br J Cancer. 2008;98:1457-1466

10. Evans DGR, Howell A. Breast cancer risk-assessment models. Breast Cancer Res. 2007;9:213

11. Sa-Nguanraksa D, Sasanakietkul T, O-Charoenrat C, Kulprom A, O-Charoenrat P. Gail Model Underestimates Breast Cancer Risk in Thai Population. Asian Pac J Cancer Prev. 2019;20(8):2385-2389 doi: 10.31557/APJCP. 2019.20.8.2385

12. Zhang L, Jie Z, Xu S, Zhang L, Guo X. Use of Receiver Operating Characteristic (ROC) Curve Analysis for Tyrer-Cuzick and Gail in Breast Cancer Screening in Jiangxi Province, China. Med Sci Monit. 2018;24:5528-5532 doi: 10.12659/MSM.910108

13. Solikhah S, Nurdjannah S.Assessment of the risk of developing breast cancer using the Gail model in Asian females. A systematic review. Heliyon. 2020;6(4):e03794. doi: 10.1016/j.heliyon.2020.e03794

14. E.K. Kim, K.H. Ko, K.K. Oh, J.Y. Kwak, J.K. You, M.J. Kim, B.W. ParkClinical application of the BI-RADS final assessment to breast sonography in conjunction with mammography. AJR Am J Roentgenol. 2018(190):1209-1215

15. Lee SH, Chung J, Choi HY. et al. Evaluation of Screening US-detected Breast Masses by Combined Use of Elastography and Color Doppler US with B-Mode US in Women with Dense Breasts: A Multicenter Prospective Study. Radiology. 2017;285(2):660-669

16. Seo Mirinae, Ahn et al. Comparison and Combination of Strain and Shear Wave Elastography of Breast Masses for Differentiation of Benign and Malignant Lesions by Quantitative Assessment: Preliminary Study[J]. J Ultrasound Med. 2017:0278-4297

17. Farrokh A, Wojcinski S, Degenhardt F. Diagnostic value of strain ratio measurement in the differentiation of malignant and benign breast lesions[in German]. Ultraschall Med. 2011;32:400-405

18. Youk JH, Son EJ, Gweon HM, Kim H, Park YJ, Kim JA. Comparison of Strain and Shear Wave Elastography for the Differentiation of Benign From Malignant Breast Lesions, Combined With B-mode Ultrasonography: Qualitative and Quantitative Assessments. Ultrasound Med Biol. 2014;40(10):2336-44

19. Koh J, Kim EK, Kim MJ, Yoon JH, Park VY, Moon HJ. Role of elastography for downgrading BI-RADS category 4a breast lesions according to risk factors. Acta Radiol. 2019;60(3):278-285 doi: 10.1177/0284185118780901. Epub 2018 Jun 11

20. Adler DD, Carson PL, Rubin J M. et al. Doppler ultrasound color flow imaging in the study of breast cancer: Preliminary findings [J]. Ultrasound in Medicine & Biology. 1990;16(6):553-559

21. Itoh A, Ueno E, Tohno E, Kamma H, Takahashi H, Shiina T, Yamakawa M, Matsumura T.'Breast disease. clinical application of US elastography for diagnosis. Radiology. 2006;239(2):341-350

22. Lynch HT, Watson P, Conway T, Fitzsimmons ML, Lynch J. Breast cancer family history as a risk factor for early onset breast cancer. Breast Cancer Res Treat. 1988;11(3):263-7 doi: 10.1007/BF01807285

23. Brewer HR, Jones ME, Schoemaker MJ. et al. Family history and risk of breast cancer: an analysis accounting for familystructure. Breast Cancer Res Treat. 2017;165(1):193-200 doi: 10.1007/s10549-017-4325-2

24. Zhang L, Huang Y, Feng Z. et al. Comparison of breast cancer risk factors among molecular subtypes: A case-only study. Cancer Med. 2019;8(4):1882-1892 doi: 10.1002/cam4.2012

25. Marie M C A, Zoe G, Michaela W. et al. Performance of Breast Cancer Risk-Assessment Models in a Large Mammography Cohort. J Natl Cancer Inst. 2020;112(5):489-497 doi: 10.1093/jnci/djz177

26. Gao F, Machin D, Chow KY. et al. Assessing risk of breast cancer in an ethnically South-East Asia population(results of a multiple ethnic groups study). BMC Cancer. 2012;12:529

27. Sa-Nguanraksa D, Sasanakietkul T, O-Charoenrat C, Kulprom A, O-Charoenrat P. Asian Pac J Cancer Prev. Gail Model Underestimates Breast Cancer Risk in Thai Population. 2019;20(8):2385-2389 doi: 10.31557/APJCP.2019.20.8.2385

28. O'Grady S, Morgan MP. Microcalcifications in breast cancer: From pathophysiology to diagnosis and prognosis. Biochim Biophys Acta Rev Cancer. 2018;1869(2):310-320 doi: 10.1016/j.bbcan.2018.04.006. Epub 2018 Apr 21

29. Sadigh G, Carlos RC, Neal CH, Dwamena BA. Accuracy of quantitative ultrasound elastography for differentiation of malignant and benign breast abnormalities: a meta-analysis. Breast Cancer Res Treat. 2012;134(3):923-931

30. Han J, Li F, Peng C. et al. Reducing Unnecessary Biopsy of Breast Lesions: Preliminary Results with Combination of Strain and Shear-Wave Elastography. Ultrasound Med Biol. 2019;45(9):2317-2327 doi: 10.1016/j.ultrasmedbio.2019.05.014.Epub 2019 Jun 17

Author contact

Corresponding address Corresponding authors: Hong-Yan Wang & Yu-Xin Jiang contributed equally to this work. Department of Ultrasound, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No.1 Shuai Fu Yuan, Dong Cheng District, Beijing 100730, China. E-mail: wanghongyancn (Hong-Yan Wang) and, jiangyuxinxhcom (Yu-Xin Jiang). Tel: +86 106 915 5494; +86 106 915 5491 and Fax: +86 106 915 5494; +86 106 915 5402 for Hong-Yan Wang & Yu-Xin Jiang, respectively.


Received 2020-7-30
Accepted 2020-10-18
Published 2021-1-1


Citation styles

APA
Niu, Z., Tian, J.W., Ran, H.T., Ren, W.D., Chang, C., Yuan, J.J., Kang, C.S., Deng, Y.B., Wang, H., Luo, B.M., Guo, S.L., Zhou, Q., Xue, E.S., Zhan, W.W., Zhou, Q., Li, J., Zhou, P., Zhang, C.Q., Chen, M., Gu, Y., Xu, J.F., Chen, W., Zhang, Y.H., Wang, H.Q., Li, J.C., Wang, H.Y., Jiang, Y.X. (2021). Risk-predicted dual nomograms consisting of clinical and ultrasound factors for downgrading BI-RADS category 4a breast lesions - A multiple centre study. Journal of Cancer, 12(1), 292-304. https://doi.org/10.7150/jca.51302.

ACS
Niu, Z.; Tian, J.W.; Ran, H.T.; Ren, W.D.; Chang, C.; Yuan, J.J.; Kang, C.S.; Deng, Y.B.; Wang, H.; Luo, B.M.; Guo, S.L.; Zhou, Q.; Xue, E.S.; Zhan, W.W.; Zhou, Q.; Li, J.; Zhou, P.; Zhang, C.Q.; Chen, M.; Gu, Y.; Xu, J.F.; Chen, W.; Zhang, Y.H.; Wang, H.Q.; Li, J.C.; Wang, H.Y.; Jiang, Y.X. Risk-predicted dual nomograms consisting of clinical and ultrasound factors for downgrading BI-RADS category 4a breast lesions - A multiple centre study. J. Cancer 2021, 12 (1), 292-304. DOI: 10.7150/jca.51302.

NLM
Niu Z, Tian JW, Ran HT, Ren WD, Chang C, Yuan JJ, Kang CS, Deng YB, Wang H, Luo BM, Guo SL, Zhou Q, Xue ES, Zhan WW, Zhou Q, Li J, Zhou P, Zhang CQ, Chen M, Gu Y, Xu JF, Chen W, Zhang YH, Wang HQ, Li JC, Wang HY, Jiang YX. Risk-predicted dual nomograms consisting of clinical and ultrasound factors for downgrading BI-RADS category 4a breast lesions - A multiple centre study. J Cancer 2021; 12(1):292-304. doi:10.7150/jca.51302. https://www.jcancer.org/v12p0292.htm

CSE
Niu Z, Tian JW, Ran HT, Ren WD, Chang C, Yuan JJ, Kang CS, Deng YB, Wang H, Luo BM, Guo SL, Zhou Q, Xue ES, Zhan WW, Zhou Q, Li J, Zhou P, Zhang CQ, Chen M, Gu Y, Xu JF, Chen W, Zhang YH, Wang HQ, Li JC, Wang HY, Jiang YX. 2021. Risk-predicted dual nomograms consisting of clinical and ultrasound factors for downgrading BI-RADS category 4a breast lesions - A multiple centre study. J Cancer. 12(1):292-304.

This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/). See http://ivyspring.com/terms for full terms and conditions.
Popup Image