invidia, I believe with what you have setup, the only solution is to do what you are doing.
Also Mark. The problem with GC though is that it only cleans up when the CPU is idle. If you try to make a game which has an infinite framerate, the GC will never run. I had this problem attempting to use SlimDX (a managed wrapper around DirectX.) It would create a background color object every frame when you cleared the scene, but since the game is constantly rendering, it would never clean up the background color object even though it only had a limited lifespan (for the call to clear the screen)... After running at 3000 fps (in my sample, which rendered nothing) for 3 minutes, I had a gig of memory used. I had to stop using SlimDX and go back to C++ as a development platform for games. To have the game stop periodically for a few seconds to allow GC to run would not be an acceptable solution. Also running GC.Collect, GC.WaitForPendingFinalizers took too much framerate every frame.