Noisy-domain NR: feasibility (write-up only, nothing built)
Back to the report · Static analysis · ControlMask how-to
Written by Claude Opus 5.5 (Claude Code, high effort) on behalf of GrimsVerk, who skimmed it. The research, the test-build code changes, the measurements and this text are Claude's work.
The idea (GrimsVerk's): run the NR edit on the noisy, jittered, internal-resolution path-traced render (before DLSS Ray Reconstruction), and output the same noisy, jittered image with the NR edit applied. RR then works as usual. NR would cost less (fewer pixels), and the final picture would be better because RR reconstructs the edit together with everything else.
Short answer: the cost argument does not hold any more. OptiScaler-NR can already run the model at internal resolution (WorkingScale 0.5 = 1920x1080 for a 4K screen) on the clean finished picture, so moving it before RR saves no model time. It adds a denoise pass. What remains is a possible quality gain: RR may enlarge the edit more faithfully than OptiScaler-NR's guided enlargement. No NVIDIA parameter or mode accepts noisy, linear or HDR input. The only realistic route is one that GrimsVerk's OptiScaler-NR fork already has (DenoiseFirst step 2). The smallest experiment is one photo-mode session with existing code.
1. What the runtimes accept (from the static analysis)
None of the controls found in either version makes NR work in a noisy or linear domain:
| Question | Answer | Evidence |
|---|---|---|
| Colour range | display-referred RGB in [0, 1] only. Encoded as (c - 0.5)/8 for the network; output clamped to [0, 1]; alpha written as 1 | pre-block / post-process PTX |
| Linear or HDR input, exposure | none. No exposure parameter, no sRGB/PQ conversion, no HDR path | parameter list, both versions |
| Noise | the network already gets its own per-frame Gaussian noise in 3 input channels. It is a generative model trained to add detail on top of a clean picture. Path-tracer noise in the colour input is just more "texture" to it | input channels 0-2 |
| Temporal | it fetches its own previous output at uv + mv x scale / size and blends with it through a learned weight. With a jittered input and jitter-free motion vectors, the history would be misaligned by the jitter every frame | pre-block PTX |
| Depth, guides | no depth (the network's depth slot is hard-wired to null); MVec only for history | CreateFeature/Evaluate |
| Lower working resolution | the network runs at whatever size Color has. The runtime's own reduced-resolution path is dead code | section 6 of static.html |
| Hidden "noisy" or "pre-upscale" mode | none. Two models, three styles, no other configs, no debug views | weight registry, descriptor list |
So the network sees noisy, jittered, linear HDR input as an out-of-range, grainy SDR picture. That is consistent with what the fork measured on 2026-10-04 in The Witcher 3.
2. What the fork already tried (do not repeat)
From the fork's history and its 2026-10-04/05 captures in The Witcher 3:
- NR directly on the noisy 1080p pre-RR frame ("Generate model before upscale"). NR edits about 60 % of the pixels, but almost only coarse tone and colour (rms 0.093 above 32 px). The fine detail it adds is tiny and temporally unstable (rms 0.016, frame-to-frame correlation 0.40). RR passes the coarse and mid bands through almost unchanged, and the final local contrast was 2 % lower with NR on.
- Conclusion then: noise makes NR skip detail synthesis; it is not RR erasing it. This is the direct version of the idea, and it does not work with NVIDIA's models.
- DenoiseFirst ("Denoise at native, then NR, then upscale", INI
DenoiseFirst*), three second steps (INIDenoiseFirstStep0/1/2):- upscale the NR'd clean image again with RR, zero jitter: softer (detail -8 % vs no NR);
- upscale it with plain DLSS SR, zero jitter: softer (-24 %) and flickers;
- edit = NR(clean) / clean as a ratio, resampled by the frame's jitter and applied to the raw jittered render, then the game's normal RR: the winner. RR kept the edits temporally stable. The ratio halved colour noise compared with adding the difference. It boosted path-tracer fireflies on dark floors 1.3-1.6x, so a firefly guard was added (
DenoiseFirstFireflyGuard, on by default).
- Finished-picture NR at reduced scale + guided residual enlargement (Transfer 5, the fork's current default approach) replaced all of that for daily use. It is cheaper and simpler, and HUD-safe with FG.
Step 2 is the idea, built. "Pre-denoise, NR on the clean image, carry the edit back onto the noisy jittered signal before RR" is exactly what it does, with a 1:1 RR pass as the denoiser.
3. Cost, honestly
Measured in this research (model only, 4K output, NR GPU timer):
| NR placement | pixels | small model | old model |
|---|---|---|---|
| finished picture, full 4K | 8.3 MP | 7.12 ms | 12.65 ms (310.8) |
| finished picture, WorkingScale 0.5 | 2.07 MP | 2.23 ms | 3.89 ms |
| before RR at internal res 1920x1080 (the idea) | 2.07 MP | 2.23 ms | 3.89 ms |
- The model costs the same in both placements at WorkingScale 0.5, because it processes the same number of pixels.
- Before-RR placement adds work: a denoise of the noisy render (in DenoiseFirst, an extra 1:1 DLSS-RR evaluation at 1080p), the edit transfer, and the firefly guard. Guided enlargement (about 0.5 ms of "surrounding work" in the GPU window logs) goes away. The extra RR pass was not measured here. RR is the most expensive DLSS feature, so expect it to cost more than the enlargement it replaces (an estimate, to be measured).
- So the idea is not a speed win any more. It can only win on picture quality: RR reconstructs the edit at 4K with its temporal super-resolution, instead of a guided enlargement from 1080p. The resolution sweep shows how much the enlargement loses: "detail lost" at 0.5 against full 4K NR is 0.22-0.67, depending on the subject (smallest on Yennefer, largest on the tavern faces and Geralt's dark scene). That is the gap the idea would have to close.
4. Routes
| Route | What it needs | Expected quality | Cost per frame | Effort | Verdict |
|---|---|---|---|---|---|
| A. An existing parameter or model | nothing | n/a | n/a | n/a | not possible: no mode accepts noisy/linear/HDR input, and no hidden model exists (section 1) |
| B. Cheap pre-denoise -> NR -> edit back onto the noisy signal (ratio) -> RR | already in the fork as DenoiseFirst step 2 (1:1 RR as the denoiser, ratio transfer, jitter resampling, firefly guard). A cheaper denoiser (a few-tap spatial/temporal filter) could replace the 1:1 RR pass, but NR's detail synthesis needs a really clean input, so a cheap denoiser probably gives the same "coarse tone only" result as raw input | possibly better detail than guided enlargement; risks: fireflies (guarded), double temporal filtering (NR history + RR history), jitter alignment, HUD-less ordering with FG (NR would run before the HUD anyway, so HUD-safe) | model as today + one extra RR (or denoise) pass - enlargement (~0.5 ms) | low: exists; needs a measurement, maybe a cheaper denoiser | worth one experiment |
| C. A new or fine-tuned model that maps noisy -> noisy+edit directly | training data: thousands of matched frames (noisy jittered internal render + guides, target = the same noisy render with the edit that NR applies to the converged frame, i.e. a teacher-student setup using NVIDIA NR as the teacher). A training pipeline, an inference engine in the fork (DirectML/D3D12 compute, < 1-2 ms at 1080p) | well below NVIDIA's model at first; the edit is generative (NR samples noise), so a student learning it from noisy inputs is hard | own network, can be small | very high (months) | not recommended |
Legal limits for route C. The NVIDIA weights are proprietary and come with NVIDIA's runtime licence. Fine-tuning them, extracting them into another engine, or redistributing them is out. Training a new network on outputs of NVIDIA's model (distillation) is a grey area: NVIDIA's SDK licences generally forbid using their software to build competing models, and the runtimes used here are game-bundled copies. Read the DLSS/NGX licence terms before doing anything public. For private experiments, B uses NVIDIA's runtime exactly as it is meant to be used, just on a different picture.
5. Recommendation and the smallest experiment
Recommendation: keep the current finished-picture NR (small model, WorkingScale 0.5, Transfer 5) for daily use. Run one experiment with the existing DenoiseFirst step 2 code to find out whether the RR-reconstructed edit beats guided enlargement enough to pay for the extra RR pass. Build nothing new unless it clearly does.
Smallest experiment (one game session, no new code): use the research build's live control and the frozen photo-mode frames from the report (Geralt, Yennefer, the tavern faces and houses).
- Reference: small model on the finished picture at WorkingScale 1.0 (the "ideal" edit).
- A: WorkingScale 0.5, Transfer 5 (finished picture).
- B: DenoiseFirst step 2 with the small model at internal resolution (1920x1080), ratio edit, firefly guard on.
- Measure, as in the report: detail lost against the reference (A against the reference is already known: 0.22-0.67), the model GPU time plus the total NR GPU window (it includes the extra RR pass), flicker in the frozen frame, and one 3-second camera pan for temporal stability.
Decision rule: B is only worth pursuing if its detail loss is clearly below A's (for example under half) at a total extra cost below about 1 ms, with no visible fireflies or ghosting in the pan. If B's loss is no better than A's, the idea is closed: NR before RR then costs more for the same picture.
Prediction (to be proven or disproven by the experiment): B will cost 1-3 ms more than A for a small detail gain, because the 1:1 RR pass that cleans the input costs about as much as the model itself. The fork's earlier step-2 captures looked good, but they were never measured against guided enlargement at equal cost.