From: stronk7 <stronk7>
Date: Wed, 13 Jun 2007 23:22:57 +0000 (+0000)
Subject: Now the tokeniser is using the new textlib->entities_to_utf8()
X-Git-Url: http://git.mjollnir.org/gw?a=commitdiff_plain;h=783fbd2234ba27aaafd013d8677c9dec7be62103;p=moodle.git

Now the tokeniser is using the new textlib->entities_to_utf8()
own function (PHP4 and PHP compatible)

Merged from MOODLE_18_STABLE
---

diff --git a/lib/tokeniserlib.php b/lib/tokeniserlib.php
index 737875b126..4cf3d4c40e 100644
--- a/lib/tokeniserlib.php
+++ b/lib/tokeniserlib.php
@@ -312,7 +312,7 @@ function tokenise_simplify($text, $overlap_cjk, $join_numbers) {
     $textlib = textlib_get_instance();
 
     // Decode entities to UTF-8
-    $text = html_entity_decode($text, ENT_QUOTES, 'UTF-8');
+    $text = $textlib->entities_to_utf8($text, true);
 
     // Lowercase
     $text = $textlib->strtolower($text);