Bitter taste could result from infections, inflammations, micronutrient deficiency, medication/ drug/ food item side effect, radiation exposure, hormonal/ endocrine issues, heavy metal poisoning etc.
With associated sore
jaw this is likely to be infectious/ inflammatory in nature. If there are no other symptoms, you could try taking multi-vitamin supplements for 3-4 weeks. If the symptoms do not improve or aggravate it is advised to consult a primary care physician for appropriate management.