Benchmarking .NET Framework 4.8 Against .NET 10: The Full Methodology

Part 1: Designing a fair comparison

The two applications

LegacyApi: ASP.NET Web API on .NET Framework 4.8, hosted on IIS Express, using Newtonsoft.Json.
ModernApi: ASP.NET Core Minimal API on .NET 10, hosted on Kestrel, using System.Text.Json.

Both expose GET /process, GET /health, GET /memory. Both load the same 1,000,000-record dataset (147 MB, generated with a fixed random seed of 42) into memory at startup.

The most important rule: src/LegacyApi/ProcessLogic.cs and src/ModernApi/ProcessLogic.cs use exactly the same algorithm, line for line. Both sort, filter, transform, and serialise the data. Each request returns about 18.9 MB of JSON. Neither version is optimised more than the other. If the code were different, we would be measuring our own changes, not the runtime.

Why an in-memory workload

/process is designed to measure computation, not I/O. The dataset loads once at startup, so no request touches the disk. If we added a database, we would end up measuring the database instead. Network round trips and query planning would hide any real runtime difference, and the result would tell us nothing about .NET.

This is a deliberate limit on the test. It also means the result does not apply to applications that depend on I/O. We come back to this point later.

The one thing we couldn't equalise

Newtonsoft.Json is the standard serialiser for Web API on .NET Framework. System.Text.Json is the default for ASP.NET Core. We could have forced both applications to use Newtonsoft, but that would test a setup nobody actually uses in production.

So this benchmark compares a realistic 2015 stack with a realistic 2026 stack. Some of the improvement comes from the serialiser. That is a real part of what migration gives you, but it means the final number is not a pure runtime comparison. We want to say this clearly instead of hiding it.

Part 2: Three measurement methods, and why three

A single measurement can be misleading. It is much harder to dismiss a result when three independent methods agree, so we built three.

Method 1: isolated CPU benchmark (BenchmarkDotNet)

This measures the sort, filter, and transform work directly, in-process, with no HTTP and no server. The CpuBound project targets both net48 and net10.0, so the exact same benchmark class compiles and runs on both runtimes.

BenchmarkDotNet handles the hard parts on its own. It runs separate jitting, pilot, and warm-up stages before it starts measuring, then runs many measured iterations with outlier detection and full statistics. This is why we did not wrap it in an extra repetition loop ourselves. Doing that would double-count results and give a wrong picture of what was measured.

Method 2: latency profile (k6, 2 virtual users)

Low enough concurrency that requests don't queue behind each other. These numbers reflect per-request cost.

Method 3: throughput profile (k6, 20 virtual users)

This test deliberately overloads the system. At this level of concurrency, the latency numbers are mostly caused by queuing time: twenty requests are competing for one CPU-bound endpoint on a 12-core machine. The useful number here is sustained requests per second.

These two profiles must be run separately. A common mistake is running one saturating load test and reporting its percentiles as latency. Those percentiles mostly describe queue depth, not how long a request takes. If you want both numbers you need both runs.

Part 3: The harness, and why it's built this way

Warm-up has to be excluded, not just performed

.NET uses tiered compilation. Code starts in a slower, unoptimised tier and gets recompiled once it proves it is used often. Connection pools, thread pools, and internal caches also need time to reach a steady state. If you start measuring from second zero, you are partly measuring startup, not real performance. The two runtimes warm up differently, so this effect does not cancel out.

Just running a warm-up phase is not enough. k6's built-in metrics, like http_req_duration and http_reqs, combine data from every scenario in the test. This means a warm-up scenario would still affect your percentiles. The fix is a custom metric that only records data during the real measurement:

import exec from 'k6/execution';
import { Trend, Counter } from 'k6/metrics';

const mainDuration = new Trend('main_duration', true);
const mainRequests = new Counter('main_requests');
const mainErrors   = new Counter('main_errors');

export function runIteration() {
  const isMain = exec.scenario.name === 'main';

  const res = http.get(`${BASE_URL}/process`);

  const passed = check(res, {
    'status is 200':     (r) => r.status === 200,
    'body is non-empty': (r) => !!r.body && r.body.length > 0,
  });

  if (isMain) {
    mainRequests.add(1);
    mainDuration.add(res.timings.duration);
    if (!passed) mainErrors.add(1);
  }
}

We then read percentiles from main_duration instead of from k6's built-in metrics.

The phase boundary needs a hard stop

There is a hidden trap here. When a scenario ends, k6 gives virtual users a grace period (30 seconds by default) to finish requests that are still running. If your measurement scenario starts the exact moment warm-up ends, the old warm-up users are still working while the new users have already started at full strength. This means you can get up to double your intended concurrency at the start of every measurement window.

This makes tail latency look worse than it is, and it does so unevenly: the slower application's requests overlap for longer, so the runtime that is already slower gets punished even more.

Two settings fix this problem: gracefulStop: '0s' on both scenarios, and a short gap between them. k6 does not allow an expression inside startTime, so we calculate the offset once, when the script starts:

const GAP = __ENV.GAP || '5s';

const MAIN_START_TIME =
  `${durationSeconds(WARMUP_DURATION) + durationSeconds(GAP)}s`;

export const options = {
  scenarios: {
    warmup: {
      executor: 'constant-vus',
      exec: 'runIteration',
      vus: VUS,
      duration: WARMUP_DURATION,
      startTime: '0s',
      gracefulStop: '0s',
    },
    main: {
      executor: 'constant-vus',
      exec: 'runIteration',
      vus: VUS,
      duration: DURATION,
      startTime: MAIN_START_TIME,
      gracefulStop: '0s',
    },
  },
  summaryTrendStats: ['avg','min','med','max','p(90)','p(95)','p(99)'],
};

gracefulStop: '0s' on the main scenario matters too. Without it, a request that runs past DURATION would still increase main_request_count, while throughput_rps still divides by the planned time window. That would quietly make throughput look higher than it really is.

The last line, summaryTrendStats, matters too. k6's default trend statistics do not include p99. This is not a bug. The value is just missing from the summary unless you ask for it. If you forget to request it, you risk reporting a percentile the tool never actually calculated.

Our runs used 30s warm-up, a 5s settle gap, and a 600s measurement window. Every result file records all three, so a reader can confirm which window a number came from.

Output has to be machine-readable by construction

If you just pipe k6's console output into a .json file, you get a human-readable text summary with the wrong file extension. It looks like JSON, but it is not. The harness prevents this mistake. It requires a real output path and writes structured fields directly:

const OUT_FILE = __ENV.OUT_FILE;
if (!OUT_FILE) {
  throw new Error('OUT_FILE env var is required.');
}

export function handleSummary(data) {
  const trend = data.metrics.main_duration;
  const p = (name) => (trend ? trend.values[name] : undefined);

  const summary = {
    target: TARGET, vus: VUS,
    warmup_duration: WARMUP_DURATION,
    warmup_gap: WARMUP_GAP,
    main_duration: DURATION,
    timestamp_utc: new Date().toISOString(),
    main_request_count: requestCount,
    main_error_count: errorCount,
    p50_ms: p('med'), p90_ms: p('p(90)'),
    p95_ms: p('p(95)'), p99_ms: p('p(99)'),
    mean_ms: p('avg'), min_ms: p('min'), max_ms: p('max'),
    throughput_rps: requestCount / durationSeconds(DURATION),
  };

  const json = JSON.stringify(summary, null, 2) + '\n';
  return { stdout: json, [OUT_FILE]: json };
}

main_request_count is included on purpose. The summary tool needs this number to decide if the percentiles can be trusted at all.

Runs alternate between targets

An i7-1270P is a 28-watt mobile processor. It throttles under sustained load. Run all your Framework tests then all your .NET 10 tests and whichever went second is measured on a hotter, slower machine.

So the suite alternates:

$order = @('modern', 'legacy', 'modern', 'legacy', 'modern', 'legacy')

This way, thermal drift during a multi-hour session affects both targets about equally, instead of favouring one over the other. There is also a 120-second cooldown between each block.

The harness refuses to mislabel a run

Twelve manual API swaps in one long session means twelve chances to record a result under the wrong runtime. A mislabelled result is worse than no result at all, because you might publish it by mistake. Both APIs expose an identifying field on /health, and the test runner checks this before every block:

$response = Invoke-RestMethod -Uri "$url/health" -TimeoutSec 10

if ($response.app -ne $expected) {
    throw "Mismatch: expected '$expected' at $url but /health reported " +
          "app='$($response.app)'. Refusing to proceed."
}

Percentiles from thin samples are suppressed

A p99 value calculated from only 78 requests is really just the second-slowest request, dressed up as a statistic. The summary tool enforces a minimum sample size:

$MinSamplesForPercentile = 1000

if ($isPercentile -and [int]$run.main_request_count -lt $MinSamplesForPercentile) {
    continue   # excluded from median/min/max for this metric
}

If every repetition for a target has too few samples, the table shows insufficient samples (n=…) instead of a number. Means and throughput are not percentiles, so they are never hidden this way.

This rule is what set our measurement duration. The slower application manages about 3.66 req/s under saturation, so clearing 1,000 samples requires at least 274 seconds of measurement. We used 600.

Every figure is a median, published with its spread

No single run is presented as the final answer. The summary tool reports the median, minimum, maximum, and the run-to-run spread as a percentage of the median. A row with 20% spread should not look the same to a reader as a row with only 1% spread.

The tool also refuses to calculate a percentage difference between legacy and modern runs if their timestamps are more than two hours apart. This protects against silently comparing measurements from different sessions, taken at different room temperatures, or after a reboot.

Machine specs are captured, not remembered

capture-environment.ps1 runs at the start of the test suite. It records the CPU, RAM, OS build, SDK versions, the .NET Framework registry release value, power settings, AC status, and every process using more than 1% CPU. If you type these specs in later from memory, the claim "we ran this on a clean machine" can quietly become false, without anyone noticing.

One honest limitation: reading the Windows 11 power-mode slider needs an undocumented API that our capture script could not call. We explain more about this in the section on problems we did not control for.

Part 4: Results

CPU-bound operation

2.03x faster, 20.8% less allocated. The confidence intervals are separated by roughly 174 ms. There is no reading of this data where the difference is noise.

Latency profile: 2 VUs, 3 repetitions each

Run-to-run spread 5.0–6.9% on Framework, 1.3–3.7% on .NET 10. Based on 3,924 legacy and 8,570 modern measured requests.

Throughput profile: 20 VUs, 2 repetitions each

Run-to-run spread 1.2–2.5% on both. Based on 4,386 legacy and 9,384 modern measured requests.

Remember: these latency numbers are mostly caused by queuing, so you cannot compare them directly to the latency-profile numbers above. This profile exists to measure throughput.

26,264 requests measured in total. Zero errors, zero empty bodies.

Part 5: Why it's faster

The most interesting result in the whole exercise isn't a timing.

BenchmarkDotNet records which CPU instruction sets the runtime exposes to the JIT compiler. On the same physical processor, in the same benchmark session:

.NET Framework 4.8    VectorSize=256

.NET 10               AVX2, AES, BMI1, BMI2, FMA, LZCNT,
                      PCLMUL, POPCNT, AvxVnni, SERIALIZE
                      VectorSize=256

Framework 4.8's JIT exposes almost none of the vector and bit-manipulation instructions that the CPU actually supports. .NET 10 exposes the full modern set. Just as important, the modern base class library is written to use these instructions. Span<T>-based sorting, vectorised comparison, SIMD string search, and hardware-accelerated JSON encoding all have fast paths that simply cannot run under Framework 4.8.

This is not a tuning problem. There is no setting that fixes it. Framework 4.8 is running on hardware it was never designed to understand. Because CPUs keep gaining new instruction sets over time, this gap grows wider instead of getting smaller.

This is also the simplest explanation for why our number stayed consistent. A speedup from one specific optimisation would show up in one measurement and disappear in others. A speedup from the compiler using better instructions everywhere shows up as a roughly constant factor instead. That is exactly what we saw: 2.03x, 2.16x, and 2.20x across three unrelated methods.

Two smaller contributors:

Serialisation. System.Text.Json is faster and allocates less than Newtonsoft.Json, and it's the default on modern .NET.

Response handling. ModernApi streams the response using chunked transfer encoding. LegacyApi buffers the full 18.9 MB and sets a Content-Length header instead. You can see this in the working set: 287 MB versus 433 MB after loading the same dataset, a difference of 33.7%.

Part 6: What we did not control for

Hosting is not equivalent. LegacyApi runs on IIS Express, a development server. Production Web API runs on full IIS, which would perform better. ModernApi runs on Kestrel, which is what it uses in production. Part of the throughput gap is probably caused by this, but we have not measured how much.

Serialisers differ, as described above. This makes the comparison realistic, but it means the serialiser is not a controlled variable.

Power configuration. The machine stayed on AC power the whole time. Windows reports the power scheme as Balanced. On modern-standby hardware like this, Windows 11 hides the High Performance scheme completely and adds a power-mode slider on top of Balanced instead. That slider was set to Best performance. We report this as something we observed, not something we measured, because the API that reads this slider is undocumented and our capture script could not call it. Either way, this setting applied equally to both runtimes, since we used an alternating run order.

This is a laptop. It has a 28-watt mobile CPU that throttles under sustained load. Alternating the run order spreads this effect across both runtimes, but it does not remove it completely.

One outlier. The first latency-profile repetition gave a .NET 10 p99 of 649 ms, compared to 484 ms and 475 ms in the other two runs. That is a 35.9% spread on that row. It happened to be the first run of the session. Using the median absorbs this outlier, and we chose to leave the result visible instead of deleting it.

The throughput profile has two repetitions, not three. The two results agree within 1.2%, but this is one repetition short of the latency profile.

One workload, one machine, one type of problem. CPU-bound, allocation-heavy, in-memory data transformation. Nothing here is a general rule about either runtime.

Part 7: Test environment

Both applications and the load generator ran on this single machine, back to back, in alternating order.

Part 8: Run it yourself

git clone https://github.com/skybridgesystems/dotnet-migration-benchmark

./scripts/run-smoke-test.ps1        # verify the harness end to end
./scripts/run-cpu-benchmarks.ps1    # BenchmarkDotNet, both runtimes
./scripts/run-full-suite.ps1        # both profiles, alternating
./scripts/summarize-results.ps1     # median/min/max tables

Requires Windows (Framework 4.8 runs nowhere else), .NET SDK 10, the .NET Framework 4.8 Developer Pack, 64-bit IIS Express, and k6.

Before you run the benchmark: close your code editor and browser, plug in your laptop, and check that Web.config has <compilation debug="false" />. Debug mode turns off JIT optimisation and would make every Framework number invalid.

If you get materially different results, open an issue. We'd rather know.

What this means for a migration decision

A ~2x improvement on CPU-bound work is realistic, without rewriting your logic. ProcessLogic.cs is identical in both applications. The gain came from the runtime, the JIT's access to modern instruction sets, and the standard library.

The memory reduction might matter more than the speed. 21% fewer allocations per operation, plus a third less working set, translates directly into fewer server instances and lower hosting costs every month.

Your gains depend on your workload. If your application waits on a database, the runtime isn't your bottleneck and migrating won't feel like 2x. Measure your own workload before building a business case on anyone's numbers, ours included.

Ask for the method. When someone quotes you a migration speedup, ask what else was running on the machine, whether warm-up was excluded, how many runs were averaged, and how many requests the percentiles came from. Most of the time nobody knows.