Bash Regex Pattern Matching: How To Use Variables Inside of Pattern Variable?
I'm trying to a) scan a string for a pattern match b) save the matches to a list c) filter the matches from the original string.
No matter the regex I use, it just won't match. I think the problem is the way I'm using a variable in the regex pattern variable, and/or the way I'm defining [space].
Suspected problem code:
trig_chars='\s\s' # Double space # Can be numbers, spaces, a-z, A-Z, or special chars.
pat_before_after="^[^\s]$trig_chars[^\s]" # Any characters before or after trig chars until space or beginning/end of query hit.
if [[ "$accum" =~ "$pat_before_after" ]] ; then # Trig char found.
Am I defining $trig_chars right in the $pat_before_after var? Am I using the vars correctly in the regex compare statement?
More context:
This is what I'm trying to do:
Example:
raw_query='A[space][space] this is a Bcd[space][space] test e$F[space][space] query [space][space] G'
hotstrings=('A' 'Bcd' 'e$F' 'G')
filtered_query='this is a test query'
From $raw_query get $hotstrings and $filtered_query.
Here's how I'm trying to do it:
Recursive search through each string character until a match is found. Remove the match. And build the filtered query var from the unmatched characters.
(I suspect this is not the way to do it, but haven't been successful getting tests to pass with other attempts.)
Here's the related code in full context:
#!/bin/bash
# Gets hotstrings from raw query.
# Returns hotstrings and filtered query, with trigger characters and hotstrings removed.
#Global vars
raw_query='' # User input thru Zenity
# trig_chars=' ' # Double space
trig_chars='\s\s' # Double space
hotstrings=() # Any chars in raw_query before/after trig chars until space or beginning/end of query hit.
filtered_query='' # Raw_query with hotstrings & trig chars removed.
pat_before_after="^[^\s]$trig_chars[^\s]" # Any characters before or after trig chars until space or beginning/end of query hit.
# Scans raw query for trig chars & hotstrings.
# Filters trig chars & hotstrings from filtered_query var.
get_hotstrings(){
# Guard clause edge case, no trigger chars defined.
if [ "$trig_chars" == "" ]; then
filtered_query="$raw_query"
return; fi
local scan_text="$raw_query"
local accum='' # Used to accumulate unmatched chars.
filtered_query='' # Blank out filtered query before scan.
while [ "$scan_text" != "" ]; do # Recursive search thru each char.
local next_char=${scan_text:0:1}
local accum="$accum$next_char"
if [[ "$accum" =~ "$pat_before_after" ]] ; then # Trig char found.
local l=${#accum}
local scan_text=${scan_text:$l} # Remove chars & hotscan_texts from scan_text.
hotstring=${BASH_REMATCH[1]} # Get hotstring from pattern match.
hotstrings+=("$hotstring") # Update hotstrings list
local accum='' # Start new accumulator.
else
filtered_query="$filtered_query$next_char" # Build the filtered query from non-matched chars.
scan_text=${scan_text:1} # remove char.
fi
done
}
################### TESTS ##################
test_messages(){
#$1=test_name
#$2=expected result.
#$3=actual result.
msg="Test name:$1
Expected result:$2
Actual result:$3"
if [ "$2" == "$3" ]; then
msg2='SUCCESS'
else
msg2='FAILED'
fi
echo "$msg"
echo "$msg2"
}
hotstring_scan_test(){
#$1=Test name
#$2=raw query
#$3=trig chars
#$4=Expected hotstrings.
#$5=Expected filtered query.
raw_query="$2"
trig_chars="$3"
get_hotstrings
r="${hotstrings[*]}"
msg1="Test name: $1
Raw query: $2
Trig chars: $3"
if [ "$r" == "$4" ]; then
msg2="SUCCESS"
else
msg2="Expected hotstring result:$4
Actual hotstring result:$r
FAILED"
fi
echo "$msg1"
echo "$msg2"
echo ""
msg1="Filtered query test:
Filtered query: $filtered_query"
if [ "$filtered_query" == "$5" ]; then
msg2="SUCCESS"
else
msg2="Expected result:$5
Actual result:$filtered_query
FAILED"
fi
echo "$msg1"
echo "$msg2"
echo ""
}
hotstring_scan_tests(){
local t_name='No trigger chars defined'
local q='Test query without hotstrings or trigger chars'
raw_query="$q"
local eq='Test query without hotstrings or trigger chars'
local tc=''
local eh=''
hotstring_scan_test "$t_name" "$q" "$tc" "$eh" "$eq"
filtered_query=''
local t_name='No trigger chars defined'
local q='Test query without hotstrings or trigger chars'
raw_query="$q"
local eq='Test query without hotstrings or trigger chars'
local tc='**'
local eh=''
hotstring_scan_test "$t_name" "$q" "$tc" "$eh" "$eq"
filtered_query=''
local t_name='Double space trigger chars'
local q=' aabc Test b Query with Hotstrings removed c'
raw_query="$q"
local eq='Test Query with Hotstrings removed'
local tc=' '
local eh='aabc b c'
hotstring_scan_test "$t_name" "$q" "$tc" "$eh" "$eq"
filtered_query=''
local t_name='Double space after trigger chars'
local q='aabc Test b Query with Hotstrings removed c '
raw_query="$q"
local eq='Test Query with Hotstrings removed'
local tc=' '
local eh='aabc b c'
hotstring_scan_test "$t_name" "$q" "$tc" "$eh" "$eq"
filtered_query=''
local t_name='Trigger chars before hotstring'
local q='**aabc Test **b Query with Hotstrings removed **c'
raw_query="$q"
local eq='Test Query with Hotstrings removed'
local tc='**'
local eh='aabc b c'
hotstring_scan_test "$t_name" "$q" "$tc" "$eh" "$eq"
filtered_query=''
local t_name='Trigger chars after hotstring'
local q='aabc** Test b** Query with Hotstrings removed c**'
raw_query="$q"
local eq='Test Query with Hotstrings removed'
local tc='**'
local eh='aabc b c'
hotstring_scan_test "$t_name" "$q" "$tc" "$eh" "$eq"
filtered_query=''
local t_name='Trigger chars before & after hotstring'
local q='aabc** Test **B$! Query with Hotstrings removed **c123'
raw_query="$q"
local eq='Test Query with Hotstrings removed'
local tc='**'
local eh='aabc B$1! c123'
hotstring_scan_test "$t_name" "$q" "$tc" "$eh" "$eq"
filtered_query=''
}
performance_test(){
start=$(($(date +%s%N)/1000000))
for i in {1..10000}; do
run_all_tests
done
end=$(($(date +%s%N)/1000000))
execution_time=$(expr $end - $start)
echo "Execution time in nano seconds: $execution_time"
}
run_all_tests(){
set -x
hotstring_scan_tests
}
run_all_tests
# performance_test
Solution 1:
According to man bash the use of " (double quote) with "$pat_before_after" will force the match of the entire string with the pattern in the variable.
In the same man bash, the pattern is a regular expression as defined in man 7 regex: I mean \s is not a placeholder for space, try [[:blank:]] or [[:space:]] instead (EDIT see comments).
a test (EDIT see comments):
trig_chars='[[:space:]][[:space:]]'
pat_before_after="^[^[:space:]]${trig_chars}[^[:space:]]"
accum='f daaa'
if [[ $accum =~ $pat_before_after ]] ; then
printf 'Match!\n'
fi
result:
Match!