asmsense

English · Türkçe

A small language model that looks at a function in a stripped binary and gives it a meaningful name and a short description.

Current status

The v4 dataset contains 228,177 functions from 341 projects across 5 optimization levels and is published on Hugging Face under the v4 config. Qwen3-8B + LoRA v6 adds anonymized Ghidra pseudocode to assembly and call context: name F1 is 0.135 on the fixed 2,000-row test sample versus 0.124 for v5 and 0.036 for the same base model without an adapter, but 0.135 on eval_115 versus 0.155 for v5. The improvement does not carry across both checks; both planned F1 targets (0.14 / 0.17) were missed. The source-aware English explanation audit is 62/294 correct (21.1%), with six judge failures; the 25% target was also missed. The model generates English/Turkish descriptions and a name; these suggestions still need review. See Results, sections 6–7. The v6 Apache-2.0 adapter and full-precision evidence are public; the historical v5 adapter remains available separately.

Open a stripped program in Ghidra or IDA and you will see hundreds of functions named FUN_00401a30. Much of reverse engineering consists of understanding and naming these functions one by one. This project aims to teach a model that first step:

input  (stripped x86-64)             target output
─────────────────────────────         ──────────────────────────────────────────────
movzx  eax, byte ptr [rsi]            name:        adler32_update
lea    rdx, [rdi + rax]               description: Computes the Adler-32 checksum
cmp    rdx, 0xfff1      ← 65521                    over the bytes of a buffer.
...                                   # TR: Tampondaki baytlar üzerinden Adler-32
                                      #     sağlama toplamı hesaplar.

Goal

Large general-purpose models can handle this task to some extent, but they are slow, expensive, and cloud-hosted. The goal is to answer this question:

Can a small model trained on purpose-built data match or even outperform large models on this task—while running on a laptop?

Three tiers are evaluated on the same test set:

Tier	Model	Role
Ceiling	Large open models (DeepSeek, GLM, MiMo, Gemma, Qwen)	The best currently achievable with off-the-shelf models
Baseline	Small open model, before fine-tuning	Starting point
This project	The same small model + LoRA on this dataset	The project's contribution

How it works

1. Data generation (cikar.py): Open-source C projects are compiled for x86-64 at different optimization levels (-O0, -O2). The assembly for each function is extracted. The ground truth—the original function name—comes directly from the source, so no manual labeling is needed. projeler.json records the commit and compiler flags used for every project.

2. Simulating a stripped binary: The output resembles what Ghidra shows for a function in a stripped binary:

   • project-local functions become sub_01a3, and global data becomes dat_0042 (the numbers are shuffled, with separate namespaces for -O0 and -O2),
   • branch targets use function-local labels such as loc_1 and loc_2,
   • external library calls (memcpy, __stack_chk_fail) and string literals (; -> "out of memory") remain visible because they are also visible in a real binary.

   If the function's real name appears in its own assembly—for example, in an error message—the row is marked sizinti and excluded from the test set.

3. Initial assembly-only baseline (taban.py): The model receives assembly and predicts a name. Later evaluations add call context; v6 additionally uses prepared, anonymized Ghidra pseudocode with the original assembly/context. Predictions are scored by token overlap (F1), so close answers such as crc32_update and update_crc receive partial credit.

loc_6:
movzx   r9d, byte ptr [rsi + rdx]
add     rdi, r9                 ← adler32_z (-O2), as the model sees it
add     rcx, rdi
inc     rdx
cmp     rax, rdx
jne     loc_6

Data

	Projects	Functions (-O0 + -O2)
Training	zlib, libpng, sqlite, lua, mbedtls, zstd, libsodium, expat, brotli, jansson, lz4, libyaml, xxhash, cJSON	16,804
Test (memorization-resistant)	tomlc17, cyaml, mu_json_x, sajs, picomatch	777 → evaluation set: 115 compiled rows, 95 unique source functions

The dataset is available on Hugging Face: krxi123/asmsense (16,804 training examples, 777 test examples, the eval_115 evaluation set, and license texts). It is also mirrored on Kaggle: krxiii/asmsense.

Scaled dataset (v4 pipeline, 5 optimization levels)

Permissively licensed (MIT, BSD, Apache-2.0, ISC, zlib) C projects selected from GitHub with olcekle.py were compiled at -O0/-O1/-O2/-O3/-Os using cikar_bin.py (real dylib + strip -x). Project build scripts were not run; files were compiled individually with clang, and files or projects that failed to compile were skipped. Rows were deduplicated by normalized assembly hash. Rows with assembly identical to validation/test data were removed from training; validation/test rows with matching (file, name) pairs in training were also removed. This does not remove every renamed or relocated vendored copy. Every row has a lisans field.

Split	Projects	Rows	Unique source functions
Training	290	196,117	72,179
Validation	23	19,770	6,667
Test (little-known, ≤200 stars, 2024+)	28	12,290	4,685

Of 473 projects, 352 produced rows and 120 failed to compile (kernel, embedded, or platform-dependent code). Only clang was used. The data is kept outside Git and can be regenerated with python3 aday_bul.py and python3 olcekle.py hepsi.

The training/test split uses disjoint root-project identifiers: rows from one root project do not appear on both sides, but vendored source families can cross that boundary. Test projects were deliberately chosen to be little-known (2-190 stars), and most were started in 2024-2025; this reduces obvious pretraining-overlap risk compared with zlib but does not prove its absence. Licenses: veri/LISANSLAR.md.

Known exception: vendored code. An independent audit of the released v6 data (rapor/arastirma/V6_VERI_DENETIMI.md, veri_denetim_v6.py) confirmed no shared root-project identifier, id or (project, file, name) key, but test project snkv embeds SQLite under its own file names while sqlite is a training project. The fixed 2,000-row test sample has 217 rows (10.85%) with names identical to SQLite training functions; valid_300 has 16 such rows (5.3%). These are name-overlap candidates, not verified identical-source counts. Historical score slices (section 6) do not establish that overlap has no effect on results; the detailed content census and its limitations are reported separately below.

Results

1. Memorization-resistant test: 5 little-known projects, 115 functions

Mean name F1 (1.0 = exact match). In the “Prefix-stripped F1” column, each project's common name prefix (cyaml_, mu_, sajs_) is removed from both sides; the model cannot infer this prefix from assembly.