Every 404 in Your Rails App Might Be Allocating 13 MB

A bot requests /wp-login.php on your Rails app. Rails can’t route it, raises ActionController::RoutingError, and returns a 404. That should cost almost nothing. On a Rails 8.1 app with a few thousand compiled templates, it can cost 13 MB of allocations and 27 ms of CPU.

A detailed report on rails/rails#58887 traces the cost to one method, ActionDispatch::ExceptionWrapper#build_backtrace. It affects every exception, but 404s from scanners are where it hurts, because they arrive in bulk and none of them can ever involve a template.

What Rails Is Doing on Every Exception

When a request raises, Rails wraps the exception in ExceptionWrapper so the error pages and logs can show a useful backtrace. Part of that job is mapping backtrace frames that point into compiled templates back to the original .erb source, so you see app/views/pages/show.html.erb:12 instead of a generated method name.

To do that mapping, build_backtrace walks every resolver in ActionView::PathRegistry.all_resolvers and calls built_templates on each. For FileSystemResolver, that method is:

@unbound_templates.values.flatten.flat_map(&:built_templates)

@unbound_templates is a Concurrent::Map, and #values copies the whole thing. The map holds one entry per template path the resolver has ever looked up, misses included. Every engine or gem that adds a view path adds another resolver, and each one caches the lookups that passed through it on the way to the resolver that had the template. So the cost grows with templates multiplied by resolvers.

On Rails 8.1 it runs twice per request with production error pages: once in DebugExceptions and once in ShowExceptions. A routing error pays both times, even though its backtrace has no template frame to map.

How Big the Cost Is

The issue author published a reproduction script: a blank Rails 8.1.4 app with 5,000 templates rendered once each (what a long-running process ends up doing) and 15 extra view paths standing in for engines. Their numbers on Ruby 4.0.7:

Their production app, with about 81,000 cached lookups across 15 resolvers, measured 10.0 MB and 16.9 ms per 404 in staging. With their patch applied, it dropped to 0.24 MB and 0.9 ms.

A scanner sending a few hundred requests a minute turns that into gigabytes of allocations per minute. Each 404 on its own looks cheap and returns fast. The damage shows up as garbage collection pressure, higher memory, and slower responses on the endpoints your real users hit.

Rails main Only Fixes Half of It

PR #58686, merged September 7, makes the backtrace lazy. That removes the ShowExceptions half of the cost. build_backtrace itself is unchanged, though, and DebugExceptions#log_error still builds it whenever rescued responses are logged, which is the default.

There’s one more wrinkle on main: ActionView::Precompiler fills these template caches at boot. A fresh process pays the full cost from its first exception instead of growing into it.

The Fix in Progress

The same author opened PR #58890, “Skip the template lookup in ExceptionWrapper when no frame is in a template.” A compiled template method keeps the template file as its source path, so the PR only walks the resolvers when at least one backtrace frame points to a file with a registered template handler extension. A routing error has no such frame, so it skips the lookup entirely and returns the backtrace as is.

With the change applied, a 404 on the 5,000-template app goes back to 0.06 MB and 0.18 ms. As of September 30 the PR is open and hasn’t been reviewed yet.

What to Do Before It Ships

Keep scanner traffic away from Rails. Paths like /wp-login.php, /.env, and /xmlrpc.php never need to reach your app. Returning 404 for them at your CDN, load balancer, or Nginx, or with a Rack middleware such as Rack::Attack mounted early in the stack, avoids the exception entirely. This is worth doing regardless of this bug.

If you’re on main or an 8.2 prerelease, setting config.action_dispatch.log_rescued_responses = false in production removes the remaining cost. You lose the log line for rescued exceptions such as 404s, so decide whether you rely on it first.

If you want the fix now on 8.1, the issue includes the patch the author runs in production. It checks whether any frame looks like a compiled template method before doing the lookup. As an initializer:

# config/initializers/exception_wrapper_backtrace.rb
# Workaround for rails/rails#58887. Remove once PR #58890 ships.
module SkipTemplateLookupWithoutTemplateFrames
  TEMPLATE_METHOD_NAME = /\A_[a-z_]*__\d+_\d+\z/

  private

  def build_backtrace
    locations = @exception.backtrace_locations || []
    return super if locations.any? { |location| TEMPLATE_METHOD_NAME.match?(location.base_label.to_s) }

    locations.dup
  end
end

ActionDispatch::ExceptionWrapper.prepend(SkipTemplateLookupWithoutTemplateFrames)

This patches a private Rails method, so treat it as temporary. Test it in staging, confirm template errors still show the .erb file and line in your error pages, and delete it when you upgrade to a release that includes the fix. A false positive in the pattern falls back to the current behavior, so the risk is that it doesn’t help, not that it breaks backtraces.

Seeing This in Production

Routing errors never reach a controller, so most APM tools won’t list them as endpoints. What you see instead is the side effect: process memory creeping up and response times drifting on endpoints that didn’t change.

Scout Monitoring tracks allocations per request and flags requests that grow process memory with memory bloat detection. If memory climbs while none of your endpoints show more allocations, the work is happening in middleware, and error handling is a good place to look. After you block scanner paths or apply the patch, the same charts show whether it worked.

For application monitoring with errors, logs, and traces, Scout Monitoring provides the fastest insights without the bloat.