Module | ScopedSearch::QueryLanguage::Parser |
In: |
lib/scoped_search/query_language/parser.rb
|
The Parser module adss methods to the query language compiler that transform a string into an abstract syntax tree, which can be used for query generation.
This module depends on the tokeinzer module to transform the string into a stream of tokens, which is more appropriate for parsing. The parser itself is a LL(1) recursive descent parser.
DEFAULT_SEQUENCE_OPERATOR | = | :and |
LOGICAL_INFIX_OPERATORS | = | [:and, :or] |
LOGICAL_PREFIX_OPERATORS | = | [:not] |
NULL_PREFIX_OPERATORS | = | [:null, :notnull] |
COMPARISON_OPERATORS | = | [:eq, :ne, :gt, :gte, :lt, :lte, :like, :unlike, :in, :notin] |
ALL_INFIX_OPERATORS | = | LOGICAL_INFIX_OPERATORS + COMPARISON_OPERATORS |
ALL_PREFIX_OPERATORS | = | LOGICAL_PREFIX_OPERATORS + COMPARISON_OPERATORS + NULL_PREFIX_OPERATORS |
Start the parsing process by parsing an expression sequence
# File lib/scoped_search/query_language/parser.rb, line 19 19: def parse 20: @tokens = tokenize 21: while @tokens.last.is_a?(Symbol) do 22: @tokens.delete_at(@tokens.size - 1) 23: end 24: parse_expression_sequence(true).simplify 25: end
Parses a comparison
# File lib/scoped_search/query_language/parser.rb, line 72 72: def parse_comparison 73: next_token if peek_token == :comma # skip comma 74: return (String === peek_token) ? parse_infix_comparison : parse_prefix_comparison 75: end
Parses a sequence of expressions
# File lib/scoped_search/query_language/parser.rb, line 28 28: def parse_expression_sequence(initial = false) 29: expressions = [] 30: next_token if !initial && peek_token == :lparen # skip staring :lparen 31: expressions << parse_logical_expression until peek_token.nil? || peek_token == :rparen 32: next_token if !initial && peek_token == :rparen # skip final :rparen 33: return ScopedSearch::QueryLanguage::AST::LogicalOperatorNode.new(DEFAULT_SEQUENCE_OPERATOR, expressions) 34: end
Parses an infix expression, i.e. <field> <operator> <value>
# File lib/scoped_search/query_language/parser.rb, line 83 83: def parse_infix_comparison 84: lhs = parse_value 85: return case peek_token 86: when nil 87: lhs 88: when :comma 89: next_token # skip comma 90: lhs 91: else 92: if COMPARISON_OPERATORS.include?(peek_token) 93: comparison_operator = next_token 94: rhs = parse_value 95: ScopedSearch::QueryLanguage::AST::OperatorNode.new(comparison_operator, [lhs, rhs]) 96: else 97: lhs 98: end 99: end 100: end
Parses a logical expression.
# File lib/scoped_search/query_language/parser.rb, line 37 37: def parse_logical_expression 38: lhs = case peek_token 39: when nil; nil 40: when :lparen; parse_expression_sequence 41: when :not; parse_logical_not_expression 42: when :null, :notnull; parse_null_expression 43: else; parse_comparison 44: end 45: 46: if LOGICAL_INFIX_OPERATORS.include?(peek_token) 47: operator = next_token 48: rhs = parse_logical_expression 49: ScopedSearch::QueryLanguage::AST::LogicalOperatorNode.new(operator, [lhs, rhs]) 50: else 51: lhs 52: end 53: end
Parses a NOT expression
# File lib/scoped_search/query_language/parser.rb, line 56 56: def parse_logical_not_expression 57: next_token # = skip NOT operator 58: negated_expression = case peek_token 59: when :not; parse_logical_not_expression 60: when :lparen; parse_expression_sequence 61: else parse_comparison 62: end 63: return ScopedSearch::QueryLanguage::AST::OperatorNode.new(:not, [negated_expression]) 64: end
Parse values in the format (val, val, val)
# File lib/scoped_search/query_language/parser.rb, line 103 103: def parse_multiple_values 104: next_token if peek_token == :lparen #skip :lparen 105: value = [] 106: value << current_token if String === next_token until peek_token.nil? || peek_token == :rparen 107: next_token if peek_token == :rparen # consume the :rparen 108: value.join(',') 109: end
Parses a set? or null? expression
# File lib/scoped_search/query_language/parser.rb, line 67 67: def parse_null_expression 68: return ScopedSearch::QueryLanguage::AST::OperatorNode.new(next_token, [parse_value]) 69: end
Parses a prefix comparison, i.e. without an explicit field: <operator> <value>
# File lib/scoped_search/query_language/parser.rb, line 78 78: def parse_prefix_comparison 79: return ScopedSearch::QueryLanguage::AST::OperatorNode.new(next_token, [parse_value]) 80: end
This can either be a constant value or a field name.
# File lib/scoped_search/query_language/parser.rb, line 112 112: def parse_value 113: if String === peek_token 114: ScopedSearch::QueryLanguage::AST::LeafNode.new(next_token) 115: elsif ([:in, :notin].include? current_token) 116: value = parse_multiple_values() 117: ScopedSearch::QueryLanguage::AST::LeafNode.new(value) 118: else 119: raise ScopedSearch::QueryNotSupported, "Value expected but found #{peek_token.inspect}" 120: end 121: end