scanning – Remove tiny colourful fibers (not dust) from scanned image

I have imaged scanned in 600 dpi. The paper is old, specially designed, with visible texture and tiny colourful fibers being part of its structure. I’d like to “extract” image from it. Removing texture is not a problem – I achieved that with color curves option. But these colour fibers still remain. How can I get rid of them?

The problem is that I try to “clean” copperplate graphics so I don’t want to remove subtle lines of drawing itself. Which filter or tool should I use?

I’m using Linux and GIMP but if you have the tips for other software (Photoshop etc.), just share your thoughts, please.

I attach two images – first is raw scan, next is the image without paper texture but with fibers still visible.

Raw scan

Partially cleaned scan