oncology.

Sequencing-based cancer studies routinely produce mutation annotation files that describe many somatic variants for each sample. These records contain useful clues about tumor type, but they are not naturally organized as a fixed-length feature vector. One patient may have only a small number of reported variants, while another may have hundreds. In addition, most individual variants occur rarely across the cohort. The modeling problem is therefore to summarize a variable-length mutation profile into a sample-level representation while preserving the information carried by individual mutation records [1], [2].

Previous work has shown that tumor type can be inferred not only from well-known driver genes but also from broader passenger mutation patterns [3]. This finding is useful because it implies that tissue-of-origin information is distributed across the mutation profile. At the same time, mutation profiles can be affected by cohort composition, sequencing depth, preprocessing choices, and class imbalance. For this reason, a reliable evaluation must avoid leakage between training and validation data and should report more than a single accuracy value.

Deep learning methods have been used to address the weak and distributed nature of mutation-derived signals. Mutation-Attention (MuAt) learns representations of mutation contexts and aggregates them for tumor typing and subtyping [2]. Multiple-instance learning is another natural choice for this setting, because the label is assigned to the sample, whereas the observations are individual mutations [1]. In both cases, the model must learn how mutation-level information should be pooled into a patient-level prediction.

Other studies have used vector-space representations of genomic alterations with classical machine-learning models [4], reduced neural networks for lung cancer histology [5], mutational signatures for primary-metastatic discrimination [6], and genome-wide mutation features for cancer of unknown primary [7], [8]. These studies support the general idea that mutation-derived features are informative, while also showing that no single representation is sufficient for all clinical and pan-cancer settings.

The clinical motivation is clearest when the tissue of origin is uncertain or when molecular profiling is the main available evidence. Related approaches have used targeted sequencing, nationwide genomic profiling, sparse liquid-biopsy mutation profiles, cfDNA methylation, and transcriptome-based assays to prioritize candidate tissues of origin [7–13]. Mutation-based models are a practical part of this broader effort because somatic variants are already produced by many cancer sequencing workflows.

Mutation profiles are also used for related prediction tasks. Microsatellite instability has been predicted from NGS data using multiple-instance learning and tabular methods such as NGBoost-based models [14], [15]. These tasks share several technical issues with tumor typing: Variable input length, high-dimensional categorical fields, class imbalance, and the risk that preprocessing statistics leak information from the validation set.

In this study, tumor-type classification from The Cancer Genome Atlas (TCGA) somatic mutations is formulated as a sequence modeling problem. Each mutation is converted into a token containing categorical and numerical descriptors, and a transformer encoder is used to model relations among the tokens within the same sample. A learnable (CLS) token is used to obtain the sample-level representation for 20-class tumor-type prediction.

The work makes three practical contributions. First, it keeps the mutation-level structure of the data instead of immediately reducing each sample to a handcrafted vector. Second, it uses stable hashed encoding for categorical variables and fold-specific normalization for numerical variables to reduce avoidable leakage. Third, it reports cross-validated accuracy, weighted accuracy, Top-k accuracy, ROC-AUC, and confusion-matrix behavior in a consistent experimental setting.

The results show that keeping mutation records as tokens is a reasonable strategy for TCGA tumor-type classification. Instead of compressing all variants into a fixed handcrafted vector before modeling, the transformer receives a structured list of mutation-level observations and learns how to combine them at the sample level through a contextualized (CLS) representation.

The input-sequence analysis clarifies why padding, truncation, and masking were needed. The cohort had a median of 101 mutation tokens per sample, but the mean was 349.2 because a smaller subset of highly mutated samples created a long right tail. Most samples required padding, while 11.2% exceeded the 512 token limit and were truncated. This shows that the model had to handle both sparse and highly mutated profiles within the same evaluation setting.

The improvement over ATGC in both accuracy and weighted accuracy suggests that the token-based representation captured information that was useful for final class assignment. The high Top-3 and Top-5 values are also important. In practice, a ranked list of candidate tumor types may be more useful than a single forced label, especially when mutation profiles are incomplete or when biologically related tumors share similar alterations.

The class-wise analysis showed substantial variation among tumor types. SKCM achieved the strongest F1- score, whereas HNSC and PRAD were more difficult. These differences are consistent with the idea that mutation-only signals are not equally distinctive across all cancer types. Some classes may require additional information, such as copy-number alteration, methylation, expression, mutational signatures, or clinical variables, to improve separation.

The confusion matrix and the ranked confusion table provide a more detailed view of the remaining errors. The largest error patterns occurred between LUAD and LUSC and between several biologically or clinically related tumor groups. Such overlap is expected in a pan-cancer mutation-only setting and may be reduced by using hierarchical classifiers or multimodal molecular inputs.

The preprocessing choices were selected to keep the evaluation conservative. Stable hashing reduced the need for a vocabulary built from all samples, and fold-specific normalization ensured that validation statistics were not used during training. These details are important in mutation-profile studies, where leakage can easily inflate performance estimates.

The findings are consistent with previous mutation-based tumor-typing studies. Multiple-instance learning treats each sample as a bag of mutation instances [1], while MuAt and related attention-based models learn representations from mutation context [2]. Other studies based on targeted sequencing, vector-space transformations, and large genomic profiling cohorts also show that mutation-derived features can support tumor classification [4], [9], [10].

The study has limitations. First, the evaluation was limited to TCGA and did not include an independent external cohort. TCGA is useful for method development, but it does not cover all sequencing platforms, sample qualities, or clinical populations. Second, the maximum sequence length was fixed at 512 mutations, which may truncate information in highly mutated tumors. Third, the model used mutation-derived features only and did not integrate other molecular or clinical information.

A further limitation is the absence of a detailed ablation study. Future experiments should measure the separate contribution of gene identity, sequence context, variant type, consequence class, VAF, read depth, hotspot status, sequence ordering, (CLS)-based sequence summarization, and class-imbalance handling. Such analysis would show which parts of the token representation are most responsible for performance.

Future work should also test the model on independent multi-center data and evaluate calibration if the outputs are used as probabilities. A hierarchical design could first predict broad tissue groups and then distinguish closely related subtypes. Combining mutation tokens with other omics layers may reduce the remaining confusion among biologically similar cancers.

This work presented a transformer-based model for tumor-type classification from somatic mutations. Each sample was represented as a sequence of mutation tokens, and a contextualized (CLS) token was used as the sample-level representation for classification. The evaluation used TCGA data after retaining tumor classes with at least 180 samples, resulting in 8,910 samples and 20 classes.

The model achieved 64.4% overall accuracy, 65.4% weighted accuracy, and 95.8% one-vs-rest ROC-AUC. It also achieved 86.7% Top-3 accuracy and 93.1% Top-5 accuracy. Class-wise and confusion-matrix analyses showed that the model performed strongly for several tumor types but still confused some related classes. Future work should focus on external validation, ablation studies, calibration, and integration of additional molecular features.

Conceptualization, S.S. and R.J.A.T.; methodology, S.S.; software, S.S.; validation, S.S. and R.J.A.T.; formal analysis, S.S.; investigation, S.S.; data curation, S.S.; writing-original draft, S.S.; writing-review and editing, S.S. and R.J.A.T.; visualization, S.S.; supervision, R.J.A.T. All authors have read and agreed to the final version of the manuscript.

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

In this study, publicly available datasets were analyzed. These data can be found in TCGA through the Genomic Data Commons (GDC) Data Portal. Processed data tables and analysis scripts are available from the corresponding author upon reasonable request.

The authors declare no conflict of interest. Funders played no role in the design of the study, in the collection, analysis, or interpretation of data, in the writing of the manuscript, or in the decision to publish the results.

This study used publicly available, de-identified genomic data and did not involve new recruitment of human participants or new collection of biospecimens. Therefore, no additional institutional ethics approval was required for the analyses reported here.