Click a task to assign a worker, Shift-click or right-click to remove.