Interacting with that thing (particularly the rotation one) is far from simple. I would have each part be a separate piece. You mentioned you have done picking so that's great, for your translation one all you have to do it when a user clicks down find out what they are clicking on. If it is the green arrow then remember the current mouse position and remember it's the green arrow. At this point you may want to transform that mouse position into world space (project it onto a plane that is parallel to the axis you selected, for green you might want to use green/red as vectors on that plane). Next when the user moves their mouse, project the new mouse position back onto the same plane. Work out the difference in whatever axis it was and move your object by that much. I foresee some weird behaviour possibly occurring depending on the plane you use, for the red/blue arrows I think it's safe to use red/blue plane but for the green if you used the green/blue plane (in that image) and dragged right horizontally then the thing would probably move down which isn't what you want. For that reason it might be an idea to use a plane that has a normal facing the user. I think you'd need to try it out or get some input off others for that.
If you want to move in a given plane (for example x/z) then you could put a little square in the corner between your arrows to click on. Use the same principle again by projecting onto the plane that the square lies on.
The rotation one is very similar, if the user clicks on the yellow circle you project their mouse onto that plane, you know the origin of the circle so given those you can work out the angle that the mouse is around it (you could use atan2). When the user drags you do the same, project the new position, work out the new angle and the difference between the two is how much to rotate your object.
Make collision meshes of sorts, the rotation one is then just a bunch of rings. I'd make them rings so they can only be interacted with first on the edges. You could ray cast to select things but since you have picking already I'd be tempted to use that.
I quite like this problem, wouldn't mind having a go at coding this one up myself :).