The short answer is no  - you can't change the face color of buttons conditionally.  You can change the text properties in the button.
Toggle Buttons are covered in Access Expert Level 4

In the Yes/No Fields video extended cut Richard goes over this pretty extensively.  
I don't see that you are a TechHelp member so that wouldn't be available to you unless you become a TechHelp member (Which I highly recommend by the way.)  There is a TON of stuff there that you typically don't get in the average website.  But then again, Richard isn't your average instructor.
You can learn more about Membership here:  JOIN

You can't apply Conditional Formatting to toggle buttons. If it's a series of individual buttons on a single form then you could use VBA to change their specific properties like color and that, but it would only work for single forms, not continuous forms.

Personally, I'm not a huge fan of toggle buttons. I prefer combo boxes.

Could you not use a rectangle control and put it behind the toggle button? Then just set the toggle to transparent.

That just might work as an alternative Dan.  I'll have to play with it!
But the original question was for the Toggle itself - that still is a no.

Unfortunately, it does not look like you are able to set the Fill/Back Color to Transparent.
The Transparent option is disabled.

Dan's idea of having a rectangle behind the toggle does have some merit though in that you could make the buttons "stand out" by formatting the rectangle underneath.  That you do have more control over.

I use a "light-up" rectangle behind buttons all the time to indicate that there are items that need attention... Like if there are Learning Connection memberships that need to be processed. It used to be a manual thing, and I didn't have them every day, so I wanted the button to "light up" to show me I needed to do it. Now, of course, it's automated, but I left the box just in case.