Modern technology offers a wide array of possibilities to publish almost any content freely on the Internet. Because of the importance and delicacy of medical information, the quality of such texts provided to general public seems to be a serious issue nowadays. Unfortunately, the only feasible way to approve the adequacy of the medical information content is human verification. Best practices in medicine are systematically captured by medical guidelines (MGL), which are provided by renowned medical societies and based on results of Evidence-Based Medicine (EBM). We propose a simple approach exploiting MGL content as a benchmark for the assessment of a content quality in medical web sites (WS). It is based on the idea that the information content or at least the scope of a medical text is reflected in the domain terminology used. We discuss a possible use of this approach in semiautomatic human-based quality verification and various aspects related to its application. Concept candidates discovered in a MGL and in the tested web pages are matched to UMLS, yielding sets of used medical terms and corresponding concepts. Several aggregation techniques for MGLs were proposed and tested. The two sets are analyzed for overall similarity at term and concept level. The method was applied on a selected medical topic employing relevant MGL and 100 WS. All the analyzed web pages fell into five distinct categories (corresponding to the target audience). Aggregations for the MGLs were proposed and tested. The average cosine similarity to MGL across all tested WS reached 0.69 whereas the average similarity calculated per each category varied up to 7,6% against the overall number. The research done is the first step towards automated evaluation of a medical web page content on the basis of MGLs as the quality standard. We describe further tasks which would improve the outputs of comparison and the possibility of its common application.