I was asked to sell one of my .com domains and I refused. But I don't have a trademark.
I am afraid that the guy will register the trademark and take it away from me.

How much does a trademark cost?

Assuming this is for the United States.

The cost by itself is about $400 assuming everything goes well. Getting a lawyer to help you with everything will be another few hundred dollars more. There are many details you need to get right -- such as sending an actual example of the mark in use -- and an experienced lawyer can help you get it correct the first time.

Since you should already be engaged in commerce to register the mark, the costs involved are generally not too bad.

Bitlaw has a page describing the relevant ICANN policies for challenging the ownership of domain name that infringes on a trademark. You might want to give it a once-over.