Great question. Yes, you do have control over changing that "Reply To" address and name. To make those changes, follow these steps:

  1. Go to the Workflow Rule Detail page of the rule that's sending your email alert
  2. Go to lower level section holding the "Workflow Actions
  3. Click on your Email Alert "Description" field. 
  4. At the very bottom of this page are two fields: Reply to Name and Reply to Address
  5. Fill in these fields, which will result in these values appears as the reply to name and address for all those who receive this alert