Found it. I was looking for something with "screenshot" in the name, but it's actually "iHiResScreenScale". Default is 4. It doesn't seem to use the full native resolution as a base though. At 720p, a value of 3 results in a 1752p screenshot. A value of 4 gives 2336p, and 5 gives 2920p, suggesting 584p as the base res instead of 720p.
6 is the highest value it accepts. Anything higher just caps to 6. To get higher screenshot resolution, I need to increase the resolution of the game. At 1080p, a value of 6 gives 10080x5664 screenshots, which look good enough when scaled back down to 1080p, without any aliasing.
Edit: Another way to increase screenshot res is to raise the internal resolution scale with fResolutionScale. Setting it to 2 at 720p with iHiResScreenScale set to 6 gives 13920x7824 screenshots.