MEDIUM: regex: Use pcre_study always when PCRE is used, regardless of JIT

pcre_study() has been around long before JIT has been added. It also seems to
affect the performance in some cases (positive).

Below I've attached some test restults. The test is based on
http://sljit.sourceforge.net/regex_perf.html (see bottom). It has been modified
to just test pcre_study vs. no pcre_study. Note: This test does not try to
match specific header it's instead run over a larger text with more and less
complex patterns to make the differences more clear.

% ./runtest
'mark.txt' loaded. (Length: 19665221 bytes)
-----------------
Regex: 'Twain'
[pcre-nostudy] time:    14 ms (2388 matches)
[pcre-study] time:    21 ms (2388 matches)
-----------------
Regex: '^Twain'
[pcre-nostudy] time:   109 ms (100 matches)
[pcre-study] time:   109 ms (100 matches)
-----------------
Regex: 'Twain$'
[pcre-nostudy] time:    14 ms (127 matches)
[pcre-study] time:    16 ms (127 matches)
-----------------
Regex: 'Huck[a-zA-Z]+|Finn[a-zA-Z]+'
[pcre-nostudy] time:   695 ms (83 matches)
[pcre-study] time:    26 ms (83 matches)
-----------------
Regex: 'a[^x]{20}b'
[pcre-nostudy] time:    90 ms (12495 matches)
[pcre-study] time:    91 ms (12495 matches)
-----------------
Regex: 'Tom|Sawyer|Huckleberry|Finn'
[pcre-nostudy] time:  1236 ms (3015 matches)
[pcre-study] time:    34 ms (3015 matches)
-----------------
Regex: '.{0,3}(Tom|Sawyer|Huckleberry|Finn)'
[pcre-nostudy] time:  5696 ms (3015 matches)
[pcre-study] time:  5655 ms (3015 matches)
-----------------
Regex: '[a-zA-Z]+ing'
[pcre-nostudy] time:  1290 ms (95863 matches)
[pcre-study] time:  1167 ms (95863 matches)
-----------------
Regex: '^[a-zA-Z]{0,4}ing[^a-zA-Z]'
[pcre-nostudy] time:   136 ms (4507 matches)
[pcre-study] time:   134 ms (4507 matches)
-----------------
Regex: '[a-zA-Z]+ing$'
[pcre-nostudy] time:  1334 ms (5360 matches)
[pcre-study] time:  1214 ms (5360 matches)
-----------------
Regex: '^[a-zA-Z ]{5,}$'
[pcre-nostudy] time:   198 ms (26236 matches)
[pcre-study] time:   197 ms (26236 matches)
-----------------
Regex: '^.{16,20}$'
[pcre-nostudy] time:   173 ms (4902 matches)
[pcre-study] time:   175 ms (4902 matches)
-----------------
Regex: '([a-f](.[d-m].){0,2}[h-n]){2}'
[pcre-nostudy] time:  1242 ms (68621 matches)
[pcre-study] time:   690 ms (68621 matches)
-----------------
Regex: '([A-Za-z]awyer|[A-Za-z]inn)[^a-zA-Z]'
[pcre-nostudy] time:  1215 ms (675 matches)
[pcre-study] time:   952 ms (675 matches)
-----------------
Regex: '"[^"]{0,30}[?!\.]"'
[pcre-nostudy] time:    27 ms (5972 matches)
[pcre-study] time:    28 ms (5972 matches)
-----------------
Regex: 'Tom.{10,25}river|river.{10,25}Tom'
[pcre-nostudy] time:   705 ms (2 matches)
[pcre-study] time:    68 ms (2 matches)

In some cases it's more or less the same but when it's faster than by a huge margin.
It always depends on the pattern, the string(s) to match against etc.

Signed-off-by: Christian Ruppert <c.ruppert@babiel.com>

diff --git a/include/common/regex.h b/include/common/regex.h
index 762eb56..58b9dc1 100644 (file)
--- a/include/common/regex.h
+++ b/include/common/regex.h
@@ -29,6 +29,12 @@
 #ifdef USE_PCRE
 #include <pcre.h>
 #include <pcreposix.h>
+
+/* For pre-8.20 PCRE compatibility */
+#ifndef PCRE_STUDY_JIT_COMPILE
+#define PCRE_STUDY_JIT_COMPILE 0
+#endif
+
 #else /* no PCRE */
 #include <regex.h>
 #endif
@@ -129,9 +135,14 @@ int regex_exec_match2(const struct my_regex *preg, char *subject, int length,
 static inline void regex_free(struct my_regex *preg) {
 #if defined(USE_PCRE) || defined(USE_PCRE_JIT)
        pcre_free(preg->reg);
-#ifdef USE_PCRE_JIT
+/* PCRE < 8.20 requires pcre_free() while >= 8.20 requires pcre_study_free(),
+ * which is easily detected using PCRE_CONFIG_JIT.
+ */
+#ifdef PCRE_CONFIG_JIT
        pcre_free_study(preg->extra);
-#endif
+#else /* PCRE_CONFIG_JIT */
+       pcre_free(preg->extra);
+#endif /* PCRE_CONFIG_JIT */
 #else
        regfree(&preg->regex);
 #endif

diff --git a/src/regex.c b/src/regex.c
index c0b23cb..760a1ed 100644 (file)
--- a/src/regex.c
+++ b/src/regex.c
@@ -288,7 +288,6 @@ int regex_comp(const char *str, struct my_regex *regex, int cs, int cap, char **
                return 0;
        }
 
-#ifdef USE_PCRE_JIT
        regex->extra = pcre_study(regex->reg, PCRE_STUDY_JIT_COMPILE, &error);
        if (!regex->extra && error != NULL) {
                pcre_free(regex->reg);
@@ -296,9 +295,6 @@ int regex_comp(const char *str, struct my_regex *regex, int cs, int cap, char **
                return 0;
        }
 #else
-       regex->extra = NULL;
-#endif
-#else
        int flags = REG_EXTENDED;
 
        if (!cs)