PDF

J Cancer 2021; 12(6):1687-1697. doi:10.7150/jca.51845 This issue Cite

Research Paper

Efficacy of second-line treatments for patients with advanced human epidermal growth factor receptor 2 positive breast cancer after trastuzumab-based treatment: a systematic review and bayesian network analysis

Fei Chen, Naifei Chen, Zheng Lv, Lingyu Li, Jiuwei Cui

Cancer Center, the First Hospital of Jilin University, Changchun, China.

✉ Corresponding author: Jiuwei Cui, E-mail: cuijwedu.cn; ORCID: 0000-0001-6496-7550.

Abstract

Purpose: Different second-line treatments of patients with trastuzumab-resistant human epidermal growth factor receptor 2 (HER2) positive breast cancer were examined in randomized controlled trials (RCTs). A network meta-analysis is helpful to evaluate the comparative survival benefits of different options.

Methods: We performed a bayesian network meta-analysis using R-4.0.0 software and fixed consistency model to compare the progression free survival (PFS) and overall survival (OS) benefits of different second-line regimens.

Results: 13 RCTs (19 publications, 4313 patients) remained for qualitative synthesis and 12 RCTs (17 publications, 4022 patients) were deemed eligible for network meta-analysis. For PFS, we divided network analysis into two parts owing to insufficient connections among treatments. The first part involved 8 treatments in 9 studies and we referred it as PFS (#1). Amid the following 8 interventions: pyrotinib + capecitabine, T-DM1 + atezolizumab, pertuzumab + trastuzumab + capecitabine, T-DM1, trastuzumab + capecitabine, lapatinib + capecitabine, neratinib, and capecitabine, we found consistent benefits between the first three interventions; moreover, pyrotinib + capecitabine was most likely to be associated with the best benefits; capecitabine monotherapy was associated with the worst PFS. The second part included 3 treatments in 2 studies and we referred it as PFS (#2): everolimus + trastuzumab + vinorelbine had better PFS benefits versus trastuzumab + vinorelbine and afatinib + vinorelbine. For OS, we analyzed 7 treatments in 7 studies, and observed T-DM1 + atezolizumab, pertuzumab + trastuzumab + capecitabine, and T-DM1 had similar effectiveness, and the first had the highest probability to yield the longest OS; capecitabine or neratinib alone yielded the worst OS benefits.

Conclusions: Our work comprehensively summarized and analyzed current available RCT-based evidence of the second-line treatments for trastuzumab-treated, HER2-positive, advanced breast cancer. These results provide clinicians and oncologists meaningful references for clinical drug administration and the development of novel effective therapies.

Keywords: human epidermal growth factor receptor 2 positive, metastatic breast cancer, advanced breast cancer, second-line treatment, network meta-analysis

Introduction

Breast cancer is the malignant tumor with the highest incidence and the second highest mortality rate worldwide for women [1]. Human epidermal growth factor receptor 2 (HER2) gene amplification or HER2 overexpression occurs in about 20% breast cancers, which is closely related to higher tumor invasiveness and shorter patients' survival [2]. The advent of HER2 monoclonal antibody trastuzumab has greatly improved the prognosis of these patients [3-5]. Numerous patients with advanced breast cancer received trastuzumab combined with chemotherapy, which was the standard first-line treatment before the approval of pertuzumab (another anti-HER2 monoclonal antibody) addition. However, primary or secondary resistance was inevitable. Hence developing the optimal second-line treatment was extremely pivotal to ameliorate long-term survival of these patients. A number of studies have explored the efficacy of multiple second-line therapeutic options, including trastuzumab cross-line therapy combined with different chemotherapeutic agents, different anti-HER2 tyrosine kinase inhibitors (TKIs) such as lapatinib, neratinib, afatinib and pyrotinib alone or combined with chemotherapy, trastuzumab plus TKIs, antibody-cytotoxic drug conjugate trastuzumab emtansine (T-DM1) alone or combined with immunotherapy, and addition of other targeted drugs such as pertuzumab or the mTOR inhibitor everolimus on the basis of trastuzumab plus chemotherapy, for HER2-positive advanced breast cancer.

However, there was a lack of head-to-head comparison between certain treatments, and the relative effects among all of these choices remained unclear. Encouragingly, the methodology of the network meta-analysis can achieve multiple treatments comparisons, that is, all direct evidence derived from randomized controlled trials (RCTs) can be statistically compared directly and indirectly in one net framework, so as to concurrently obtain pairwise comparisons of all included interventions and calculate ranking probabilities of each treatment [6]. Previous network meta-analysis regarding the second-line treatments for trastuzumab-treated HER2-positive advanced breast cancer only partly compared treatments but did not incorporate lately available trials or alternative treatments [7]. Consequently, we performed this updated systematic review and bayesian network meta-analysis to comprehensively summarize and compare relative survival benefits of different second-line treatment strategies, tested in RCTs, for trastuzumab-treated advanced breast cancer, with the purpose of providing assistance for clinical decision-making and prolonging patients' survival.

Methods

Search strategy

We followed the PRISMA (preferred reporting items for systematic reviews and meta-analyses) extension statement for network meta-analysis [8]. A literature search was conducted up to October 6, 2020, through PubMed search engine, PubMed Central (PMC), Embase and Cochrane Central Register of Controlled Trials (CENTRAL) electronic databases. The terms used for searching potential reports included “HER2-positive” or its variables, “metastatic breast cancer” or “advanced breast cancer”, trastuzumab or trastuzumab-resistant or trastuzumab-refractory or trastuzumab-containing or trastuzumab-based, and terms relevant to RCTs. All of these were restricted by field identifiers and combined by appropriate Boolean operators (Supplementary Table S2). To avoid omissions, we also checked the reference lists of relevant articles for additional publications.

Inclusion criteria

We included studies that met the following criteria: (1) involved patients with cytological or histologically confirmed HER2-positive breast cancer; (2) compared two or more treatments in second-line setting for trastuzumab-treated HER2-positive breast cancer; (3) reported endpoints: progression free survival (PFS), and (or) overall survival (OS); (4) were designed as RCTs. Articles or abstracts that did not meet any above criteria were excluded.

Data extraction

One author (F. Chen) extracted the main characteristics including trial name and its sample size, patients' median age, treatment regimens, and primary endpoints. Another author (N. F. Chen) confirmed the results. If relevant articles reported the same cohort of patients, only the update results were considered. If the same endpoint was evaluated by both independent review committee (IRC) and researchers, we extracted the data evaluated by the IRC. A senior reviewer (J. W. Cui) would make a final decision if there was any disagreement.

Quality assessment

Risk of bias of individual study was assessed by the Cochrane Risk of Bias Tool imbedded in the Review Manager (version 5.3) which bases on the following facets: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective outcome reporting, and other sources of bias. Items were marked as low, high, or unclear risk of bias.

Statistical analyses

We generated network diagrams for different outcomes by the Stata software (version 15.1, Stata, Corp, College Station, TX), to elucidate the direct and indirect comparisons among different treatments in the included studies [9]. Network meta-analyses of PFS and OS were conducted within a bayesian framework, which is more accurate than frequentist approaches [10], using Markov Chain Monte Carlo methods with the help of “gemtc” (version 0.8.4) and “rjags” (version 4.1.0) package of R-4.0.0 software. Hazard ratio (HR) and corresponding 95% credible interval (CrI) were used to assess the comparative efficacy between two treatments. The I² statistic was used to demonstrate the heterogeneity of included studies, with I² ≤50% denotes no or low heterogeneity and fixed effects model was applied, while I² >50% indicates obvious heterogeneity and the random effects model was used. With three Markov chains, 250000 sample iterations were generated with 50000 burn-ins and a thinning interval of 1 in both PFS and OS analyses. We visually inspected the trace plot and density plot that showed the fit of the three chains to evaluate the convergence of iterations, and conformed to the Brooks-Gelman-Rubin diagnosis [11]. The posterior ranking probability of each treatment was established by calculating the surface under the cumulative ranking (SUCRA) value, which equals 0 when an intervention is definite to be the worst, and larger value indicates higher likelihood of a given treatment being better [10]. We assessed global inconsistency by comparing the fit of consistency and inconsistency models [12], and also applied the node-splitting method to detect the local inconsistency in any closed loops, with P < 0.05 denotes the existence of inconsistency between direct and indirect evidence [13, 14]. We also performed sensitivity analyses by changing the effects model. Additionally, for studies that were not eligible for network meta-analysis, their data were summarized narratively using a qualitative data synthesis approach.

Results

Search results

The literature search totally led to 3492 records. Through removing duplicates, then screening titles and abstracts, total 45 promising publications were fully read. According to predefined inclusion criteria, finally 13 RCTs (19 publications) involving 4313 patients remained for qualitative synthesis and 12 RCTs (4022 patients) were deemed eligible for network meta-analysis (Fig. 1).

Main characteristics and quality evaluation

Including literature consisted of 16 journal articles and 3 conference abstracts related to 13 RCTs. 13 different treatments included 1) trastuzumab + capecitabine, 2) lapatinib + capecitabine, 3) pyrotinib + capecitabine, 4) pertuzumab + trastuzumab + capecitabine, 5) trastuzumab + vinorelbine, 6) afatinib + vinorelbine, 7) everolimus + trastuzumab + vinorelbine, 8) trastuzumab + lapatinib, 9) T-DM1, 10) T-DM1 + atezolizumab, 11) neratinib, 12) lapatinib, and 13) capecitabine. 12 studies reported the PFS data and 9 studies reported the OS data. Table 1 lists the primary features of all included studies. Cochrane Risk of Bias Tool was used to assess the quality of all studies (Fig. 2).

 Figure 1 

Flowchart of the study selection process.

Network meta-analyses

Due to the limitations of treatments comparisons, we divided the PFS analysis into two parts. The first part involved 8 treatments in 9 studies [15, 16, 20, 22, 24, 26-29] and we referred it as PFS (#1) and the second part included 3 treatments in 2 studies [30, 31] and we referred it as PFS (#2). In terms of OS, we analyzed 7 treatments in 7 studies [15, 16, 18, 21, 23, 25, 29]. Fig. 3 showed the network maps of direct and indirect comparisons among included studies. As the I² value was 10%, 50%, and 23% in included studies for PFS (#1), PFS (#2) and OS, respectively, we applied fixed consistency model for network meta-analyses.

Progression free survival

Firstly, we compared the relative efficacy of the following treatments: 1) trastuzumab + capecitabine, 2) lapatinib + capecitabine, 3) pyrotinib + capecitabine, 4) pertuzumab + trastuzumab + capecitabine, 5) T-DM1, 6) T-DM1 + atezolizumab, 7) neratinib, and 8) capecitabine, of 9 trials. The results illustrated that pyrotinib + capecitabine yielded greater PFS benefits than T-DM1 (HR 0.80, 95% CrI 0.68 to 0.95), trastuzumab + capecitabine (0.73, 0.58 to 0.92), lapatinib + capecitabine (0.66, 0.57 to 0.77), neratinib (0.61, 0.51 to 0.74), and capecitabine (0.50, 0.43 to 0.57). No significant difference was observed among pyrotinib + capecitabine, T-DM1 + atezolizumab, and pertuzumab + trastuzumab + capecitabine, as the HR values crossed 1.00. T-DM1 + atezolizumab was more beneficial than lapatinib + capecitabine (0.76, 0.62 to 0.92), neratinib (0.70, 0.56 to 0.87), and capecitabine (0.56, 0.45 to 0.71). Pertuzumab + trastuzumab + capecitabine outperformed neratinib (0.77, 0.61 to 0.97) and capecitabine (0.62, 0.49 to 0.79), and had a tendency of surpassing trastuzumab + capecitabine (0.91, 0.83 to 1.01). T-DM1 showed higher benefit in prolonging PFS when compared with lapatinib + capecitabine (0.83, 0.76 to 0.89), neratinib (0.76, 0.66 to 0.88) and capecitabine (0.62, 0.53 to 0.71). Consistent efficacy was found among lapatinib + capecitabine, trastuzumab + capecitabine, and neratinib in providing PFS benefits, and all of which yielded longer PFS than capecitabine monotherapy (Table 2).

Secondly, we compared 3 treatments (trastuzumab + vinorelbine, afatinib + vinorelbine, and everolimus + trastuzumab + vinorelbine) of 2 trials. It was found that everolimus + trastuzumab + vinorelbine had greater PFS benefit than trastuzumab + vinorelbine (HR 0.90, 95% CrI 0.83 to 0.97) and afatinib plus vinorelbine (0.86, 0.75 to 0.99). No significant difference was observed between the treatments of vinorelbine plus trastuzumab or plus afatinib (Table 3).

Overall survival

In terms of OS benefit, the network analysis demonstrated that T-DM1 + atezolizumab was better than lapatinib + capecitabine (HR 0.77, 95% CrI 0.60 to 1.00), capecitabine (0.74, 0.56 to 0.97), and neratinib (0.70, 0.52 to 0.96). Pertuzumab + trastuzumab + capecitabine was associated with longer OS, compared with trastuzumab + capecitabine (0.89, 0.80 to 0.99) and capecitabine (0.83, 0.69 to 0.99). T-DM1 was more efficacious than lapatinib + capecitabine (0.88, 0.82 to 0.95), capecitabine (0.84, 0.75 to 0.94), and neratinib (0.80, 0.66 to 0.97). Consistent OS benefit was observed among T-DM1 + atezolizumab, pertuzumab + trastuzumab + capecitabine, and T-DM1 (HRs strode across 1.00). There was no significant difference between trastuzumab + capecitabine, lapatinib + capecitabine, capecitabine, and neratinib (Table 4).

 Figure 2 

Risk of bias of included studies.

 Figure 3 

Network plots of comparisons on different outcomes. Every circular node represents a kind of treatment. The node diameter is proportional to the sample size of patients receiving a treatment. Each line represents a kind of head-to-head comparison. The width of line is proportional to the number of trials.

 Table 2 

Relative effect sizes of PFS (#1) benefit calculated from network meta-analysis

According to the order of the surface under the cumulative ranking curve (SUCRA) value from largest to smallest, the treatments are ranked from upper left to lower right. Numbers in blue boxes are SUCRA values. Numbers in gray boxes are HRs and their 95% CrIs of column-defining treatments versus row-defining treatments. HRs less than 1.00 favor the column-defining treatments. Significant pairwise comparisons are highlighted in bold. Pyro: pyrotinib; Cape: capecitabine; Atezo: atezolizumab; Pertu: pertuzumab; Trastu: trastuzumab; Lapa: lapatinib.

 Table 3 

Relative effect sizes of PFS (#2) benefit calculated from network meta-analysis

According to the order of the SUCRA value from largest to smallest, the treatments are ranked from upper left to lower right. Numbers in blue boxes are SUCRA values. Numbers in gray boxes are HRs and their 95% CrIs of column-defining treatments versus row-defining treatments. HRs less than 1.00 favor the column-defining treatments. Significant pairwise comparisons are highlighted in bold.

 Table 4 

Relative effect sizes of OS benefit calculated from network meta-analysis

According to the order of the surface under the cumulative ranking curve (SUCRA) value from largest to smallest, the treatments are ranked from upper left to lower right. Numbers in blue boxes are SUCRA values. Numbers in gray boxes are HRs and their 95% CrIs of column-defining treatments versus row-defining treatments. HRs less than 1.00 favor the column-defining treatments. Significant pairwise comparisons are highlighted in bold. Atezo: atezolizumab; Pertu: pertuzumab; Trastu: trastuzumab; Cape: capecitabine; Lapa: lapatinib.

Rank probabilities

The SUCRA values of interventions for each outcome were calculated to demonstrate their posterior ranking orders, as listed in Tables 2-4. In the second-line setting for trastuzumab-treated HER2-positive advanced breast cancer, with respect to PFS (#1) benefit, pyrotinib + capecitabine had the highest probability of ranking first (SUCRA = 0.972), followed by T-DM1 + atezolizumab (SUCRA = 0.810), pertuzumab + trastuzumab + capecitabine (SUCRA = 0.665), and T-DM1 (SUCRA = 0.649). In terms of PFS (#2) benefit, everolimus + trastuzumab + vinorelbine was the best option (SUCRA = 0.990). In terms of OS, T-DM1 + atezolizumab was likely to be ranked first (SUCRA = 0.915), followed by pertuzumab + trastuzumab + capecitabine (SUCRA = 0.778) and T-DM1 (SUCRA = 0.739).

Inconsistency assessment and sensitivity analyses

Using deviance information criteria (DIC), we observed the fit of the consistency model was similar or superior than that of inconsistency model, with smaller DIC values in consistency model (Supplementary Table S3). There was no inconsistency in direct and indirect effects of treatments within the closed loops in both PFS (#1) and OS network, because the node splitting analysis did not indicate significant differences in comparisons (P > 0.05) (Supplementary Fig. S1). We conducted sensitivity analyses of PFS and OS by replacing effects model. It was found that the DIC values calculated by the random effects model were close to that calculated by the fixed effects model, which proved the reliability of our analyses (Supplementary Table S4).

Discussion

To date, there are a variety of options in the second-line setting for trastuzumab-treated HER2-positive advanced breast cancer. We systematically reviewed the survival results of direct comparisons between treatments in RCTs and indirectly compared these treatments by network meta-analysis method. In terms of PFS, amid the following 8 treatments: pyrotinib + capecitabine, T-DM1 + atezolizumab, pertuzumab + trastuzumab + capecitabine, T-DM1, trastuzumab + capecitabine, lapatinib + capecitabine, neratinib, and capecitabine, we found consistent benefits between the first three interventions. Moreover, pyrotinib + capecitabine was most likely to be associated with the best benefits. Capecitabine monotherapy was associated with the worst PFS. In addition, everolimus + trastuzumab + vinorelbine was superior than trastuzumab or afatinib plus vinorelbine. In terms of OS, we observed that T-DM1 + atezolizumab, pertuzumab + trastuzumab + capecitabine, and T-DM1 had similar effectiveness, and the first was most likely to yield the longest OS. Capecitabine alone and neratinib alone yielded the worst OS benefits. Our work provided meaningful references for clinical drug administration and the development of novel effective therapies.

In our first part PFS analysis, we found pyrotinib plus capecitabine was the most beneficial option in second-line setting for advanced HER2-positive breast cancer patients that failed first-line trastuzumab-based treatments. The excellent efficacy of pyrotinib plus capecitabine may be mainly attributed to the pharmacological mechanism of pyrotinib. Pyrotinib is a pan-target TKI that irreversibly inhibits HER1, HER2, and HER4 sites [34] and has shown satisfying antitumor activity in phase I clinical trials [35, 36]. Comparing the PFS data of RCTs horizontally, it was found that pyrotinib combined with capecitabine brought the longest PFS, although this cross-trial comparison was risky. The phase III PHENIX study [27] showed that the patients taking pyrotinib combined with capecitabine had a PFS of 11.1 months, which was 7 months longer than the capecitabine group. The phase II Fei Ma et al. study [26] illustrated that pyrotinib plus capecitabine was more beneficial than lapatinib plus capecitabine for PFS (not reached versus 7.1 months), and the phase III PHOEBE study [28] was performed based on this result. Among trastuzumab-resistant patients in PHOEBE, improved PFS in the pyrotinib plus capecitabine group was also observed compared with the lapatinib plus capecitabine group (12.5 months versus 6.9 months) [28]. Though above-mentioned studies are all from China and OS are not mature, pyrotinib plus capecitabine is a promising second-line therapy for trastuzumab-treated HER2-positive advanced breast cancer.

We certified T-DM1 could yield significant superior benefits of both PFS and OS over lapatinib + capecitabine, neratinib, and capecitabine. T-DM1 is an antibody-cytotoxic drug conjugate that can deliver chemotherapeutic drug maytansine to HER2 overexpressing tumor cells, thereby improving antitumor effect and reducing the harm to normal tissues [37-39]. The EMILIA study [22, 23] established the importance of T-DM1 as second-line treatment for trastuzumab-treated HER2-positive breast cancer. The remarkable anti-tumor effect of T-DM1 has also been proven in other studies. Cohort 1 of the single-arm, international multicenter phase IIIB KAMILLA study explored the efficacy of T-DM1 monotherapy for patients with HER2-positive advanced breast cancer who had previously treated with anti-HER2 agents and chemotherapy. The results suggested that T-DM1 was effective in all lines setting [40]. In the phase III THE3RESA study, HER2-positive advanced breast cancer patients who had previously received trastuzumab and lapatinib were assigned to the T-DM1 group and the doctor's choice treatment group (47% crossed to the T-DM1 group). The OS of patients in the T-DM1 group was 22.7 months, compared with 15.8 months OS in the control group (HR 0.68, 95% confidence interval [CI] 0.54 to 0.85) [41]. Similarly, retrospective studies also demonstrated the clinical benefits of T-DM1 for advanced breast cancer patients that were previously treated with trastuzumab and pertuzumab [42]. Therefore, T-DM1 is a good choice in second-line setting for trastuzumab-treated advanced HER2-positive breast cancer when it is available.

The phase II KATE2 study [29] tested the addition of immunotherapy to T-DM1 in HER2-positive advanced breast cancer that had progressed after trastuzumab-based treatment. Atezolizumab, an immune checkpoint inhibitor against programmed death-ligand 1 (PD-L1), combined with nanoparticle albumin-bound paclitaxel has demonstrated remarkable activity in PD-L1 positive metastatic triple-negative breast cancer [43]. However, in KATE2, the addition of atezolizumab to T-DM1 did not present a statistically or clinically meaningful improvement in PFS for PD-L1 non-selective population, but PD-L1 positive patients had clinically longer PFS (8.5 months versus 4.1 months) [29]. Other studies concerning immunotherapy of HER2-positive advanced breast cancer suggested that PD-L1 positive population could derive benefit from immune checkpoint inhibitors [44, 45]. From our analyses, it was found that the combination of T-DM1 and atezolizumab ranked higher than other treatments in either PFS and OS benefits (except pyrotinib + capecitabine). Consequently, further study of T-DM1 in combination with atezolizumab is deserved in HER2-positive and PD-L1-positive advanced breast cancer patients. Immune-related biomarkers may be helpful to select patients sensitive to treatments. We look forward the final OS results of KATE2.

Pertuzumab is an anti-HER2 antibody that binds to the HER2 extracellular domain, preventing the formation of HER2 homodimers and HER2/HER3 heterodimers and thus exerting antitumor effects [46]. Adding pertuzumab on the basis of trastuzumab and chemotherapy have shown superior efficacy in neoadjuvant [47], adjuvant [48] and first-line [49, 50] therapies. We found that second-line treatment of pertuzumab plus trastuzumab plus capecitabine for trastuzumab-treated HER2-positive advanced breast cancer was a little disappointed for PFS, while it was associated with the second highest OS benefit. As observed in the PHEREXA study [24, 25], dual-targeted therapy of pertuzumab and trastuzumab combined with chemotherapy only prolonged PFS by 2 months while substantially increased OS by 9 months, compared with trastuzumab plus chemotherapy. The underlying cause of discrepancy between the PFS and OS results was unknown; however, addition of pertuzumab to trastuzumab and capecitabine resulted in considerable clinically meaningful increase in OS that reached more than 3 years. Therefore, the magnitude of OS improvement supports clinicians to prescribe pertuzumab plus trastuzumab plus capecitabine for trastuzumab-treated HER2-positive advanced breast cancer patients.

Preclinical studies have shown that the mTOR inhibitor everolimus could improve the antitumor effects of trastuzumab plus vinorelbine [51, 52]. Our analysis further verified that everolimus combined with trastuzumab and vinorelbine significantly improved PFS compared to trastuzumab or afatinib plus vinorelbine. mTOR, a downstream protein of the PI3K/Akt signaling pathway, regulates transcription and translation by phosphorylating downstream proteins such as pS6, resulting in trastuzumab resistance [53]. The tumor suppressor phosphatase and tensin homologue (PTEN) can inhibit PI3K/Akt/mTOR signal transduction, and the down-regulation of PTEN gene leads to continuous activation of this signal pathway, which is one of the mechanisms of trastuzumab resistance [54, 55]. The BOLERO-3 study proved that addition of everolimus to trastuzumab and vinorelbine could reverse the resistance caused by PTEN deletion or inactivation. Additionally, the PFS subgroup analysis based on biomarkers showed that patients with low PTEN expression (HR 0.40, 95% CI 0.20 to 0.82) and high pS6 expression (HR 0.48, 95% CI 0.24 to 0.96) significantly benefited from everolimus-containing regimen [30]. Therefore, the detection of biomarkers is essential to predict the effectiveness of the everolimus-containing treatment. It is expected that the OS data of the BOLERO-3 study will further clarify the meaning of everolimus addition.

The EGF104900 study was not eligible to be included in the PFS and OS network analyses, because one of the necessary conditions for network meta-analysis is that multiple treatments must form a net framework. The EGF104900 study [32, 33] compared two chemo-free regimens, trastuzumab + lapatinib versus lapatinib. In the trastuzumab plus lapatinib group (n = 146), the risk of disease progression and death were both reduced by 26%, compared with the lapatinib group (n = 145) (for PFS: HR 0.74, 95% CI 0.58 to 0.94; for OS: 0.74, 0.57 to 0.97). Notably, 53% patients in the lapatinib group crossed to the trastuzumab plus lapatinib group after disease progression and the recalculated HR value of OS was 0.65 (95% CI 0.46 to 0.94) after excluding these crossed patients. For the same reason as the EGF104900 study, the LUX-Breast1 study was not eligible in the OS network analysis as well. This trial suggested that afatinib plus vinorelbine was inferior than trastuzumab plus vinorelbine for OS benefit (HR 1.48, 95% CI 1.12 to 1.95) [31]. The network meta-analysis suggested capecitabine alone and neratinib monotherapy were less effective than other treatments. Taken together, the efficacy of chemotherapy alone (capecitabine) and TKI monotherapy (neratinib or lapatinib) in patients with advanced breast cancer is unsatisfactory. Combination therapy or a single agent with a more comprehensive and powerful mechanism of action, such as T-DM1, is the trend for the treatment for HER2-positive advanced breast cancer. Further, trastuzumab cross-line therapy yielded better efficacy than single-agent chemotherapy, which might be attributed to that trastuzumab not only inhibits HER2 signaling but also exerts antibody-dependent cell-mediated cytotoxicity [56].

There are several limitations exist in our work. First, although only RCTs were included, methodological heterogeneity across studies was one of the unavoidable confounding factors. Network meta-analyses are similar to observational studies, and the findings obtained from pooled analyses still need to be verified by RCTs. Second, there were differences in the definition of trastuzumab resistance in different studies. Third, the interpretation of OS results should be cautious as later-line treatments exerted great influences on OS. And some trials are ongoing and their OS data have not been reached. Forth, most direct evidence was from one trial and most treatments were compared indirectly in the present network. Due to insufficient connections among all treatments, we had to separately analyzed two frameworks of PFS. Fifth, we did not assess the publication bias because of the limited number of included trials in each comparison. Sixth, we did not make subgroup analyses, such as brain metastasis status, patient population and so on, due to data sparseness across trials. Additionally, our work was only aimed at HER2-positive breast cancer patients who failed first-line treatment of trastuzumab plus chemo-agents. However, for patients taking trastuzumab- and pertuzumab-based dual-targeted anti-HER2 therapy combined with chemotherapy (current standard first-line strategy), the second-line treatment is still an unmet field that needs to be explored.

In conclusion, this systematic review and network meta-analysis comprehensively summarized and analyzed current available evidence of the second-line treatments, evaluated in RCTs, for trastuzumab-treated, HER2-positive, advanced breast cancer. Despite the limitations of our work, it helps clinicians choose the most suitable regimen for individual patient from various options, and provides meaningful references for the development of novel therapies for HER2-positive breast cancer. We expect updated data of relevant studies to further complement or update our results. Well-designed RCTs that compare top-ranked treatments are warranted to clarify efficacy differences. Furthermore, several new drugs such as tucatinib [57-61], margetuximab (MGAH22) [62-64] and antibody-conjugated drug trastuzumab deruxtecan (DS-8201) [65, 66] all have shown encouraging efficacy and safety in patients who progressed after receiving multiple anti-HER2 treatments. In the future, relevant researches can be carried out to determine the benefits degree of these drugs for HER2-positive advanced breast cancer that failed first-line trastuzumab-based treatments [67], thereby optimizing the second-line treatment strategies for these patients and improving their clinical outcomes.

Abbreviations

HER2: human epidermal growth factor receptor 2; RCT: randomized controlled trial; PFS: progression free survival; OS: overall survival; HR: hazard ratio; CrI: credible interval; T-DM1: trastuzumab emtansine; mTOR: mammalian target of rapamycin; PRISMA: preferred reporting items for systematic reviews and meta-analyses; IRC: independent review committee; NR: not reached; NA: not available; TTP: time to progression; ORR: overall response rate; DIC: deviance information criteria; PI3K: phosphatidylinositide 3-kinase; PKB: protein kinase B; PTEN: phosphatase and tensin homologue; CI: confidence interval; Pyro: pyrotinib; Cape: capecitabine; Atezo: Atezolizumab; Pertu: pertuzumab; Trastu: trastuzumab; Lapa: lapatinib.

Supplementary Material

Supplementary figures and tables.

Acknowledgements

We are grateful to all authors involved in this study and the support of funds (Jilin Provincial Department of Science and Technology (20170622011JC); Jilin Provincial Department of Science and Technology (20180101009JC); Jilin Provincial Department of Science and Technology (20190303146SF); Jilin Province Department of Finance (2018SCZWSZX-010); General Program of National Natural Science Foundation of China (81874052); General Program of National Natural Science Foundation of China (81672275).

Competing Interests

The authors have declared that no competing interest exists.

References

1. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2020. CA Cancer J Clin. 2020;70:7-30

2. Owens MA, Horten BC, Da Silva MM. HER2 amplification ratios by fluorescence in situ hybridization and correlation with immunohistochemistry in a cohort of 6556 breast cancer tissues. Clin Breast Cancer. 2004;5:63-9

3. Ross JS, Slodkowska EA, Symmans WF, Pusztai L, Ravdin PM, Hortobagyi GN. The HER-2 receptor and breast cancer: ten years of targeted anti-HER-2 therapy and personalized medicine. Oncologist. 2009;14:320-68

4. Dawood S, Broglio K, Buzdar AU, Hortobagyi GN, Giordano SH. Prognosis of women with metastatic breast cancer by HER2 status and trastuzumab treatment: an institutional-based review. J Clin Oncol. 2010;28:92-8

5. Wolff AC, Hammond MEH, Hicks DG, Dowsett M, McShane LM, Allison KH. et al. Recommendations for human epidermal growth factor receptor 2 testing in breast cancer: American Society of Clinical Oncology/College of American Pathologists clinical practice guideline update. Arch Pathol Lab Med. 2014;138:241-56

6. Cipriani A, Higgins JP, Geddes JR, Salanti G. Conceptual and technical challenges in network meta-analysis. Ann Intern Med. 2013;159:130-7

7. Paracha N, Reyes A, Diéras V, Krop I, Pivot X, Urruticoechea A. Evaluating the clinical effectiveness and safety of various HER2-targeted regimens after prior taxane/trastuzumab in patients with previously treated, unresectable, or metastatic HER2-positive breast cancer: a systematic review and network meta-analysis. Breast Cancer Res Treat. 2020;180:597-609

8. Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C. et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann Intern Med. 2015;162:777-84

9. Chaimani A, Higgins JP, Mavridis D, Spyridonos P, Salanti G. Graphical tools for network meta-analysis in STATA. PloS one. 2013;8:e76654

10. Salanti G, Ades AE, Ioannidis JP. Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. J Clin Epidemiol. 2011;64:163-71

11. Brooks SP, Gelman A. General methods for monitoring convergence of iterative simulations. J Comput Graph Stat. 1998;7:434-55

12. Dias S, Welton NJ, Sutton AJ, Caldwell DM, Lu G, Ades AE. NICE DSU technical support document 4: inconsistency in networks of evidence based on randomised controlled trials. NICE Decision Support Unit. 2014;2011:updated April 2014. http://nicedsu.org.uk/

13. van Valkenhoef G, Dias S, Ades AE, Welton NJ. Automated generation of node-splitting models for assessment of inconsistency in network meta-analysis. Res Synth Methods. 2016;7:80-93

14. Dias S, Welton NJ, Caldwell DM, Ades AE. Checking consistency in mixed treatment comparison meta-analysis. Stat Med. 2010;29:932-44

15. Takano T, Tsurutani J, Takahashi M, Yamanaka T, Sakai K, Ito Y. et al. A randomized phase II trial of trastuzumab plus capecitabine versus lapatinib plus capecitabine in patients with HER2-positive metastatic breast cancer previously treated with trastuzumab and taxanes: WJOG6110B/ELTOP. Breast. 2018;40:67-75

16. Martin M, Bonneterre J, Geyer CE Jr, Ito Y, Ro J, Lang I. et al. A phase two randomised trial of neratinib monotherapy versus lapatinib plus capecitabine combination therapy in patients with HER2+ advanced breast cancer. Eur J Cancer. 2013;49:3763-72

17. von Minckwitz G, du Bois A, Schmidt M, Maass N, Cufer T, de Jongh FE. et al. Trastuzumab beyond progression in human epidermal growth factor receptor 2-positive advanced breast cancer: a german breast group 26/breast international group 03-05 study. J Clin Oncol. 2009;27:1999-2006

18. von Minckwitz G, Schwedler K, Schmidt M, Barinoff J, Mundhenke C, Cufer T. et al. Trastuzumab beyond progression: overall survival analysis of the GBG 26/BIG 3-05 phase III study in HER2-positive breast cancer. Eur J Cancer. 2011;47:2273-81

19. Geyer CE, Forster J, Lindquist D, Chan S, Romieu CG, Pienkowski T. et al. Lapatinib plus capecitabine for HER2-positive advanced breast cancer. N Engl J Med. 2006;355:2733-43

20. Cameron D, Casey M, Press M, Lindquist D, Pienkowski T, Romieu CG. et al. A phase III randomized comparison of lapatinib plus capecitabine versus capecitabine alone in women with advanced breast cancer that has progressed on trastuzumab: updated efficacy and biomarker analyses. Breast Cancer Res Treat. 2008;112:533-43

21. Cameron D, Casey M, Oliva C, Newstat B, Imwalle B, Geyer CE. Lapatinib plus capecitabine in women with HER-2-positive advanced breast cancer: Final survival analysis of a phase III randomized trial. Oncologist. 2010;15:924-34

22. Verma S, Miles D, Gianni L, Krop IE, Welslau M, Baselga J. et al. Trastuzumab emtansine for HER2-positive advanced breast cancer. N Engl J Med. 2012;367:1783-91

23. Dieras V, Miles D, Verma S, Pegram M, Welslau M, Baselga J. et al. Trastuzumab emtansine versus capecitabine plus lapatinib in patients with previously treated HER2-positive advanced breast cancer (EMILIA): a descriptive analysis of final overall survival results from a randomised, open-label, phase 3 trial. Lancet Oncol. 2017;18:732-42

24. Urruticoechea A, Rizwanullah M, Im SA, Ruiz ACS, Lang I, Tomasello G. et al. Randomized phase III trial of trastuzumab plus capecitabine with or without pertuzumab in patients with human epidermal growth factor receptor 2-positive metastatic breast cancer who experienced disease progression during or after trastuzumab-based therapy. J Clin Oncol. 2017;35:3030-8

25. Urruticoechea A, Rizwanullah M, Im SA, Carlos A, Ruiz S, Lang I. et al. Final overall survival (OS) analysis of PHEREXA: a randomized phase III trial of trastuzumab (H) + capecitabine (X) ± pertuzumab (P) in patients with HER2- positive metastatic breast cancer (MBC) who experienced disease progression during or after H-based therapy. J Clin Oncol. 2018;36(Suppl 1):1013

26. Ma F, Ouyang Q, Li W, Jiang Z, Tong Z, Liu Y. et al. Pyrotinib or lapatinib combined with capecitabine in HER2-positive metastatic breast cancer with prior taxanes, anthracyclines, and/or trastuzumab: a randomized, phase II study. J Clin Oncol. 2019;37:2610-9

27. Jiang Z, Yan M, Hu X, Zhang Q, Ouyang Q, Feng J. et al. Pyrotinib combined with capecitabine in women with HER2+ metastatic breast cancer previously treated with trastuzumab and taxanes: a randomized phase III study. J Clin Oncol. 2019;37(Suppl 15):1001

28. Xu B, Yan M, Ma F, Hu XC, Feng JF, Ouyang Q. et al. Pyrotinib or lapatinib plus capecitabine for HER2+ metastatic breast cancer (PHOEBE): a randomized phase III trial. J Clin Oncol. 2020;38(Suppl):1003

29. Emens LA, Esteva FJ, Beresford M, Saura C, De Laurentiis M, Kim SB. et al. Trastuzumab emtansine plus atezolizumab versus trastuzumab emtansine plus placebo in previously treated, HER2-positive advanced breast cancer (KATE2): a phase 2, multicentre, randomised, double-blind trial. Lancet Oncol. 2020;21:1283-95

30. Andre F, O'Regan R, Ozguroglu M, Toi M, Xu B, Jerusalem G. et al. Everolimus for women with trastuzumab-resistant, HER2-positive, advanced breast cancer (BOLERO-3): a randomised, double-blind, placebo-controlled phase 3 trial. Lancet Oncol. 2014;15:580-91

31. Harbeck N, Huang CS, Hurvitz S, Yeh DC, Shao Z, Im SA. et al. Afatinib plus vinorelbine versus trastuzumab plus vinorelbine in patients with HER2-overexpressing metastatic breast cancer who had progressed on one previous trastuzumab treatment (LUX-Breast 1): an open-label, randomised, phase 3 trial. Lancet Oncol. 2016;17:357-66

32. Blackwell KL, Burstein HJ, Storniolo AM, Rugo H, Sledge G, Koehler M. et al. Randomized study of Lapatinib alone or in combination with trastuzumab in women with ErbB2-positive, trastuzumab-refractory metastatic breast cancer. J Clin Oncol. 2010;28:1124-30

33. Blackwell KL, Burstein HJ, Storniolo AM, Rugo HS, Sledge G, Aktan G. et al. Overall survival benefit with lapatinib in combination with trastuzumab for patients with human epidermal growth factor receptor 2-positive metastatic breast cancer: final results from the EGF104900 Study. J Clin Oncol. 2012;30:2585-92

34. Li X, Yang C, Wan H, Zhang G, Feng J, Zhang L. et al. Discovery and development of pyrotinib: A novel irreversible EGFR/HER2 dual tyrosine kinase inhibitor with favorable safety profiles for the treatment of breast cancer. Eur J Pharm Sci. 2017;110:51-61

35. Ma F, Li Q, Chen S, Zhu W, Fan Y, Wang J. et al. Phase I study and biomarker analysis of pyrotinib, a novel irreversible pan-erbb receptor tyrosine kinase inhibitor, in patients with human epidermal growth factor receptor 2-positive metastatic breast cancer. J Clin Oncol. 2017;35:3105-12

36. Li Q, Guan X, Chen S, Yi Z, Lan B, Xing P. et al. Safety, efficacy, and biomarker analysis of pyrotinib in combination with capecitabine in HER2-positive metastatic breast cancer patients: a phase I clinical trial. Clin Cancer Res. 2019;25:5212-20

37. Lewis Phillips GD, Li G, Dugger DL, Crocker LM, Parsons KL, Mai E. et al. Targeting HER2-positive breast cancer with trastuzumab-DM1, an antibody-cytotoxic drug conjugate. Cancer Res. 2008;68:9280-90

38. Junttila TT, Li G, Parsons K, Phillips GL, Sliwkowski MX. Trastuzumab-DM1 (T-DM1) retains all the mechanisms of action of trastuzumab and efficiently inhibits growth of lapatinib insensitive breast cancer. Breast Cancer Res Treat. 2011;128:347-56

39. Recondo Jr G, de la Vega M, Galanternik F, Diaz-Canton E, Leone BA, Leone JP. Novel approaches to target HER2-positive breast cancer: trastuzumab emtansine. Cancer Manag Res. 2016;8:57-65

40. Montemurro F, Ellis P, Anton A, Wuerstlein R, Delaloge S, Bonneterre J. et al. Safety of trastuzumab emtansine (T-DM1) in patients with HER2-positive advanced breast cancer: primary results from the KAMILLA study cohort 1. Eur J Cancer. 2019;109:92-102

41. Krop IE, Kim SB, Martin AG, LoRusso PM, Ferrero JM, Badovinac-Crnjevic T. et al. Trastuzumab emtansine versus treatment of physician's choice in patients with previously treated HER2-positive metastatic breast cancer (TH3RESA): final overall survival results from a randomised open-label phase 3 trial. Lancet Oncol. 2017;18:743-54

42. Dzimitrowicz H, Berger M, Vargo C, Hood A, Abdelghany O, Raghavendra AS. et al. T-DM1 activity in metastatic human epidermal growth factor receptor 2-positive breast cancers that received prior therapy with trastuzumab and pertuzumab. J Clin Oncol. 2016;34:3511-7

43. Schmid P, Rugo HS, Adams S, Schneeweiss A, Barrios CH, Iwata H. et al. Atezolizumab plus nab-paclitaxel as first-line treatment for unresectable, locally advanced or metastatic triple-negative breast cancer (IMpassion130): updated efficacy results from a randomised, double-blind, placebo-controlled, phase 3 trial. Lancet Oncol. 2020;21:44-59

44. Chia S, Bedard PL, Hilton J, Amir E, Gelmon K, Goodwin R. et al. A phase Ib trial of durvalumab in combination with trastuzumab in HER2-positive metastatic breast cancer (CCTG IND.229). Oncologist. 2019;24:1439-45

45. Loi S, Giobbie-Hurder A, Gombos A, Bachelot T, Hui R, Curigliano G. et al. Pembrolizumab plus trastuzumab in trastuzumab-resistant, advanced, HER2-positive breast cancer (PANACEA): a single-arm, multicentre, phase 1b-2 trial. Lancet Oncol. 2019;20:371-82

46. Adams CW, Allison DE, Flagella K, Presta L, Clarke J, Dybdal N. et al. Humanization of a recombinant monoclonal antibody to produce a therapeutic HER dimerization inhibitor, pertuzumab. Cancer Immunol Immunother. 2006;55:717-27

47. Gianni L, Pienkowski T, Im Y-H, Roman L, Tseng L-M, Liu M-C. et al. Efficacy and safety of neoadjuvant pertuzumab and trastuzumab in women with locally advanced, inflammatory, or early HER2-positive breast cancer (NeoSphere): a randomised multicentre, open-label, phase 2 trial. Lancet Oncol. 2012;13:25-32

48. Piccart M, Procter M, Fumagalli D, de Azambuja E, Clark E, Ewer MS. et al. Abstract GS1-04: Interim overall survival analysis of APHINITY (BIG 4-11): A randomized multicenter, double-blind, placebo-controlled trial comparing chemotherapy plus trastuzumab plus pertuzumab versus chemotherapy plus trastuzumab plus placebo as adjuvant therapy in patients with operable HER2-positive early breast cancer. Cancer Res. 2020;80(Suppl 4):GS1-04

49. Swain SM, Baselga J, Kim S-B, Ro J, Semiglazov V, Campone M. et al. Pertuzumab, trastuzumab, and docetaxel in HER2-positive metastatic breast cancer. N Engl J Med. 2015;372:724-34

50. Swain SM, Miles D, Kim SB, Im YH, Im SA, Semiglazov V. et al. Pertuzumab, trastuzumab, and docetaxel for HER2-positive metastatic breast cancer (CLEOPATRA): end-of-study results from a double-blind, randomised, placebo-controlled, phase 3 study. Lancet Oncol. 2020;21:519-30

51. Mondesire WH, Jian W, Zhang H, Ensor J, Hung M-C, Mills GB. et al. Targeting mammalian target of rapamycin synergistically enhances chemotherapy-induced cytotoxicity in breast cancer cells. Clin Cancer Res. 2004;10:7031-42

52. Zhu Y, Zhang X, Liu Y, Zhang S, Liu J, Ma Y. et al. Antitumor effect of the mTOR inhibitor everolimus in combination with trastuzumab on human breast cancer stem cells in vitro and in vivo. Tumor Biol. 2012;33:1349-62

53. Steelman LS, Chappell WH, Abrams SL, Kempf CR, Long J, Laidler P. et al. Roles of the Raf/MEK/ERK and PI3K/PTEN/Akt/mTOR pathways in controlling growth and sensitivity to therapy-implications for cancer and aging. Aging (Albany NY). 2011;3:192-222

54. Nagata Y, Lan K-H, Zhou X, Tan M, Esteva FJ, Sahin AA. et al. PTEN activation contributes to tumor inhibition by trastuzumab, and loss of PTEN predicts trastuzumab resistance in patients. Cancer cell. 2004;6:117-27

55. Pandolfi PP. Breast cancer - loss of PTEN predicts resistance to treatment. N Engl J Med. 2004;351:2337-8

56. Musolino A, Naldi N, Bortesi B, Pezzuolo D, Capelletti M, Missale G. et al. Immunoglobulin G fragment C receptor polymorphisms and clinical efficacy of trastuzumab-based therapy in patients with HER-2/neu-positive metastatic breast cancer. J Clin Oncol. 2008;26:1789-96

57. Murthy RK, Loi S, Okines A, Paplomata E, Hamilton E, Hurvitz SA. et al. Tucatinib, trastuzumab, and capecitabine for HER2-positive metastatic breast cancer. N Engl J Med. 2020;382:597-609

58. Murthy R, Borges VF, Conlin A, Chaves J, Chamberlain M, Gray T. et al. Tucatinib with capecitabine and trastuzumab in advanced HER2-positive metastatic breast cancer with and without brain metastases: a non-randomised, open-label, phase 1b study. Lancet Oncol. 2018;19:880-8

59. Lin NU, Borges V, Anders C, Murthy RK, Paplomata E, Hamilton E. et al. Intracranial efficacy and survival with tucatinib plus trastuzumab and capecitabine for previously treated HER2-positive breast cancer with brain metastases in the HER2CLIMB trial. J Clin Oncol. 2020;38:2610-19

60. Lin NU, Murthy RK, Anders CK, Borges VF, Hurvitz SA, Loi S. et al. Tucatinib versus placebo added to trastuzumab and capecitabine for patients with previously treated HER2+ metastatic breast cancer with brain metastases (HER2CLIMB). J Clin Oncol. 2020;38(Suppl 15):1005

61. Okines AFC, Paplomata E, Wahl TA, Wright GLS, Sutherland S, Jakobsen E. et al. Management of adverse events in patients with HER2+ metastatic breast cancer treated with tucatinib, trastuzumab, and capecitabine (HER2CLIMB). J Clin Oncol. 2020;38(Suppl 15):1043

62. Rugo HS, Im S-A, Wright GLS, Escriva-de-Romani S, DeLaurentiis M, Cortes J. et al. SOPHIA primary analysis: A phase 3 (P3) study of margetuximab (M)+ chemotherapy (C) versus trastuzumab (T)+ C in patients (pts) with HER2+ metastatic (met) breast cancer (MBC) after prior anti-HER2 therapies (Tx). J Clin Oncol. 2019;37(Suppl 15):1000

63. Rugo HS, Im S-A, Cardoso F, Cortes J, Curigliano G, Pegram MD. et al. GS1-02. Phase 3 SOPHIA study of margetuximab + chemotherapy vs trastuzumab + chemotherapy in patients with HER2+ metastatic breast cancer after prior anti-HER2 therapies: second interim overall survival analysis. Cancer Res. 2020;80(Suppl 4):GS1-02

64. Escrivá S, Im S-A, Cardoso F, Cortes J, Curigliano G, Gradishar WJ. et al. SOPHIA analysis by chemotherapy (Ctx) choice: A phase III (P3) study of margetuximab (M)+ Ctx versus trastuzumab (T)+ Ctx in patients (pts) with pretreated HER2+ metastatic (met) breast cancer (MBC). J Clin Oncol. 2020;38(Suppl 15):1040

65. Krop IE, Saura C, Yamashita T, Park YH, Kim S-B, Tamura K. et al. Abstract GS1-03:[Fam-] trastuzumab deruxtecan (T-DXd; DS-8201a) in subjects with HER2-positive metastatic breast cancer previously treated with T-DM1: a phase 2, multicenter, open-label study (DESTINY-Breast01). Cancer Res. 2020;80(Suppl 4):GS1-03

66. Modi S, Andre F, Krop IE, Saura C, Yamashita T, Kim S-B. et al. Trastuzumab deruxtecan for HER2-positive metastatic breast cancer: DESTINY-Breast01 subgroup analysis. J Clin Oncol. 2020;38(Suppl 15):1036

67. Verma S, Shahidi J, Lee C, Wang K, Cortes J. Phase III study of [fam-] trastuzumab deruxtecan (DS-8201a) vs T-DM1 for HER2-positive breast cancer. Ann Oncol. 2019;30(Suppl 3):iii62

Author contact

Corresponding author: Jiuwei Cui, E-mail: cuijwedu.cn; ORCID: 0000-0001-6496-7550.

Received 2020-8-11
Accepted 2020-12-13
Published 2021-1-18

Citation styles

©2024 Ivyspring International Publisher. Terms of use