Photoshop :: CS6 Very Sluggish After Assigned RAM Used Up?

during the evaluation of my PS CS6 trial I encountered the following problem. Painting on a layer mask is very, very sluggish once Photoshop CS6 has used up all RAM that has been assigned to it.When I am in 100% view and <shift><click> about 10 times in quick succession it takes almost a minute for photoshop to catch up, often the last clicks get lost and the next click draws a line without using <shift>.
 
I tried without improvement:

- Different brushes (hard, soft, down to 10px)
- Disable graphics processor
- Disable layer icons
 
The problem seems to be related to virtual memory handling. Does PS CS6 use virtual memory buffering? The Activity Monitor shows still free memory.In comparison and on the same system, Photoshop CS4 with default VM Buffering remains quite responsive although the RAM usage of PS CS4 (active + inactive) is much lower than in CS6. 
 
Image data
 
25130 x 9250 px, 16bpc, 95 layers with masks, but 92 of them with non-transparent pixels in only a small area (177 x 236 px)
  
System data
 
Adobe Photoshop Version: 13.0 (13.0 20120315.r.428 2012/03/15:21:00:00) x64
Operating System: Mac OS 10.6.8
System architecture: Intel CPU Family:6, Model:44, Stepping:2 with MMX, SSE Integer, SSE FP, SSE2, SSE3, SSE4.1, SSE4.2, HyperThreading
Physical processor count: 6
Logical processor count: 12
Processor speed: 3330 MHz
Built-in memory: 12288 MB

[code]....