class Crass::Parser

Parses a CSS string or list of tokens.

  1. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#parsing

Constants

BLOCK_END_TOKENS
DEFAULT_MAXIMUM_DEPTH

Default maximum nesting depth for simple blocks and functions. This is far higher than any legitimate CSS needs, but far below the depth at which Ruby would raise ‘SystemStackError` while recursively parsing nested constructs.

Keeping this low also bounds memory usage: each nested simple block and function retains a ‘:tokens` array spanning its descendants for serialization, so the total serialization metadata grows with nesting depth. A modest limit prevents deeply nested (but otherwise valid) input from amplifying memory disproportionately.

It can be overridden with the ‘:maximum_depth` option.

Attributes

tokens[R]

{TokenScanner} wrapping the tokens generated from this parser’s input.

Public Class Methods

new(input, options = {}) click to toggle source

Initializes a parser based on the given input, which may be a CSS string or an array of tokens.

See {Crass.parse} for options.

# File lib/crass/parser.rb, line 140
def initialize(input, options = {})
  unless input.kind_of?(Enumerable)
    input = Tokenizer.tokenize(input, options)
  end

  @depth = 0
  @maximum_depth = options[:maximum_depth] || DEFAULT_MAXIMUM_DEPTH
  @tokens = TokenScanner.new(input)
end
parse_properties(input, options = {}) click to toggle source

Parses CSS properties (such as the contents of an HTML element’s ‘style` attribute) and returns a parse tree.

See {Crass.parse} for options.

5.3.6. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#parse-a-list-of-declarations

# File lib/crass/parser.rb, line 39
def self.parse_properties(input, options = {})
  Parser.new(input, options).parse_properties
end
parse_rules(input, options = {}) click to toggle source

Parses CSS rules (such as the content of a ‘@media` block) and returns a parse tree. The only difference from {parse_stylesheet} is that CDO/CDC nodes (`<!–` and `–>`) aren’t ignored.

See {Crass.parse} for options.

5.3.3. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#parse-a-list-of-rules

# File lib/crass/parser.rb, line 50
def self.parse_rules(input, options = {})
  parser = Parser.new(input, options)
  rules  = parser.consume_rules

  rules.map do |rule|
    if rule[:node] == :qualified_rule
      parser.create_style_rule(rule)
    else
      rule
    end
  end
end
parse_stylesheet(input, options = {}) click to toggle source

Parses a CSS stylesheet and returns a parse tree.

See {Crass.parse} for options.

5.3.2. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#parse-a-stylesheet

# File lib/crass/parser.rb, line 68
def self.parse_stylesheet(input, options = {})
  parser = Parser.new(input, options)
  rules  = parser.consume_rules(:top_level => true)

  rules.map do |rule|
    if rule[:node] == :qualified_rule
      parser.create_style_rule(rule)
    else
      rule
    end
  end
end
stringify(nodes, options = {}) click to toggle source

Converts a node or array of nodes into a CSS string based on their original tokenized input.

Options:

* **:exclude_comments** - When `true`, comments will be excluded.
# File lib/crass/parser.rb, line 88
def self.stringify(nodes, options = {})
  nodes  = [nodes] unless nodes.is_a?(Array)
  string = String.new

  nodes.each do |node|
    next if node.nil?

    case node[:node]
    when :at_rule
      string << '@'
      string << node[:name]
      string << self.stringify(node[:prelude], options)

      if node[:block]
        string << '{' << self.stringify(node[:block], options) << '}'
      else
        string << ';'
      end

    when :comment
      string << node[:raw] unless options[:exclude_comments]

    when :simple_block
      string << node[:start]
      string << self.stringify(node[:value], options)
      string << node[:end]

    when :style_rule
      string << self.stringify(node[:selector][:tokens], options)
      string << '{' << self.stringify(node[:children], options) << '}'

    else
      if node.key?(:raw)
        string << node[:raw]
      elsif node.key?(:tokens)
        string << self.stringify(node[:tokens], options)
      end
    end
  end

  string
end

Public Instance Methods

consume_at_rule(input = @tokens) click to toggle source

Consumes an at-rule and returns it.

5.4.2. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-at-rule

# File lib/crass/parser.rb, line 153
def consume_at_rule(input = @tokens)
  rule = {}

  rule[:tokens] = input.collect do
    rule[:name]    = input.consume[:value]
    rule[:prelude] = []

    while token = input.consume
      node = token[:node]

      if node == :comment # Non-standard.
        next

      elsif node == :semicolon
        break

      elsif node === :'{'
        # Note: The spec says the block should _be_ the consumed simple
        # block, but Simon Sapin's CSS parsing tests and tinycss2 expect
        # only the _value_ of the consumed simple block here. I assume I'm
        # interpreting the spec too literally, so I'm going with the
        # tinycss2 behavior.
        rule[:block] = consume_simple_block(input)[:value]
        break

      elsif node == :simple_block && token[:start] == '{'
        # Note: The spec says the block should _be_ the simple block, but
        # Simon Sapin's CSS parsing tests and tinycss2 expect only the
        # _value_ of the simple block here. I assume I'm interpreting the
        # spec too literally, so I'm going with the tinycss2 behavior.
        rule[:block] = token[:value]
        break

      else
        input.reconsume
        rule[:prelude] << consume_component_value(input)
      end
    end
  end

  create_node(:at_rule, rule)
end
consume_component_value(input = @tokens) click to toggle source

Consumes a component value and returns it, or ‘nil` if there are no more tokens.

5.4.6. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-a-component-value

# File lib/crass/parser.rb, line 200
def consume_component_value(input = @tokens)
  return nil unless token = input.consume

  case token[:node]
  when :'{', :'[', :'('
    consume_simple_block(input)

  when :function
    if token.key?(:name)
      # This is a parsed function, not a function token. This step isn't
      # mentioned in the spec, but it's necessary to avoid re-parsing
      # functions that have already been parsed.
      token
    else
      consume_function(input)
    end

  else
    token
  end
end
consume_declaration(input = @tokens) click to toggle source

Consumes a declaration and returns it, or ‘nil` on parse error.

5.4.5. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-a-declaration

# File lib/crass/parser.rb, line 225
def consume_declaration(input = @tokens)
  declaration = {}
  value       = []

  declaration[:tokens] = input.collect do
    declaration[:name] = input.consume[:value]

    next_token = input.peek

    while next_token && next_token[:node] == :whitespace
      input.consume
      next_token = input.peek
    end

    unless next_token && next_token[:node] == :colon
      # Parse error.
      #
      # Note: The spec explicitly says to return nothing here, but Simon
      # Sapin's CSS parsing tests expect an error node.
      return create_node(:error, :value => 'invalid')
    end

    input.consume

    until input.peek.nil?
      value << consume_component_value(input)
    end
  end

  # Look for !important.
  important_tokens = value.reject {|token|
    node = token[:node]
    node == :whitespace || node == :comment || node == :semicolon
  }.last(2)

  if important_tokens.size == 2 &&
      important_tokens[0][:node] == :delim &&
      important_tokens[0][:value] == '!' &&
      important_tokens[1][:node] == :ident &&
      important_tokens[1][:value].downcase == 'important'

    declaration[:important] = true
    excl_index = value.index(important_tokens[0])

    # Technically the spec doesn't require us to trim trailing tokens after
    # the !important, but Simon Sapin's CSS parsing tests expect it and
    # tinycss2 does it, so we'll go along with the cool kids.
    value.slice!(excl_index, value.size - excl_index)
  else
    declaration[:important] = false
  end

  declaration[:value] = value
  create_node(:declaration, declaration)
end
consume_declarations(input = @tokens, options = {}) click to toggle source

Consumes a list of declarations and returns them.

By default, the returned list may include ‘:comment`, `:semicolon`, and `:whitespace` nodes, which is non-standard.

Options:

* **:strict** - Set to `true` to exclude non-standard `:comment`,
  `:semicolon`, and `:whitespace` nodes.

5.4.4. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-a-list-of-declarations

# File lib/crass/parser.rb, line 292
def consume_declarations(input = @tokens, options = {})
  declarations = []

  while token = input.consume
    case token[:node]

    # Non-standard: Preserve comments, semicolons, and whitespace.
    when :comment, :semicolon, :whitespace
      declarations << token unless options[:strict]

    when :at_keyword
      # When parsing a style rule, this is a parse error. Otherwise it's
      # not.
      input.reconsume
      declarations << consume_at_rule(input)

    when :ident
      decl_tokens = [token]

      while next_token = input.peek
        break if next_token[:node] == :semicolon
        decl_tokens << consume_component_value(input)
      end

      if decl = consume_declaration(TokenScanner.new(decl_tokens))
        declarations << decl
      end

    else
      # Parse error (invalid property name, etc.).
      #
      # Note: The spec doesn't say we should append anything to the list of
      # declarations here, but Simon Sapin's CSS parsing tests expect an
      # error node.
      declarations << create_node(:error, :value => 'invalid')
      input.reconsume

      while next_token = input.peek
        break if next_token[:node] == :semicolon
        consume_component_value(input)
      end
    end
  end

  declarations
end
consume_function(input = @tokens) click to toggle source

Consumes a function and returns it.

5.4.8. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-a-function

# File lib/crass/parser.rb, line 342
def consume_function(input = @tokens)
  @depth += 1

  begin
    # Discard functions nested more deeply than the maximum allowed depth to
    # avoid exhausting the Ruby stack on maliciously nested input.
    return discard_block(input) if @depth > @maximum_depth

    function = {
      :name   => input.current[:value],
      :value  => [],
      :tokens => [input.current] # Non-standard, used for serialization.
    }

    function[:tokens].concat(input.collect {
      while token = input.consume
        case token[:node]
        when :')'
          break

        # Non-standard.
        when :comment
          next

        else
          input.reconsume
          function[:value] << consume_component_value(input)
        end
      end
    })

    create_node(:function, function)
  ensure
    @depth -= 1
  end
end
consume_qualified_rule(input = @tokens) click to toggle source

Consumes a qualified rule and returns it, or ‘nil` if a parse error occurs.

5.4.3. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-a-qualified-rule

# File lib/crass/parser.rb, line 383
def consume_qualified_rule(input = @tokens)
  rule = {:prelude => []}

  rule[:tokens] = input.collect do
    while true
      unless token = input.consume
        # Parse error.
        #
        # Note: The spec explicitly says to return nothing here, but Simon
        # Sapin's CSS parsing tests expect an error node.
        return create_node(:error, :value => 'invalid')
      end

      if token[:node] == :'{'
        # Note: The spec says the block should _be_ the consumed simple
        # block, but Simon Sapin's CSS parsing tests and tinycss2 expect
        # only the _value_ of the consumed simple block here. I assume I'm
        # interpreting the spec too literally, so I'm going with the
        # tinycss2 behavior.
        rule[:block] = consume_simple_block(input)[:value]
        break
      elsif token[:node] == :simple_block && token[:start] == '{'
        # Note: The spec says the block should _be_ the simple block, but
        # Simon Sapin's CSS parsing tests and tinycss2 expect only the
        # _value_ of the simple block here. I assume I'm interpreting the
        # spec too literally, so I'm going with the tinycss2 behavior.
        rule[:block] = token[:value]
        break
      else
        input.reconsume
        rule[:prelude] << consume_component_value(input)
      end
    end
  end

  create_node(:qualified_rule, rule)
end
consume_rules(flags = {}) click to toggle source

Consumes a list of rules and returns them.

5.4.1. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-a-list-of-rules

# File lib/crass/parser.rb, line 424
def consume_rules(flags = {})
  rules = []

  while token = @tokens.consume
    case token[:node]
      # Non-standard. Spec says to discard comments and whitespace, but we
      # keep them so we can serialize faithfully.
    when :comment, :whitespace
      rules << token

    when :cdc, :cdo
      unless flags[:top_level]
        @tokens.reconsume
        rule = consume_qualified_rule
        rules << rule if rule
      end

    when :at_keyword
      @tokens.reconsume
      rule = consume_at_rule
      rules << rule if rule

    else
      @tokens.reconsume
      rule = consume_qualified_rule
      rules << rule if rule
    end
  end

  rules
end
consume_simple_block(input = @tokens) click to toggle source

Consumes and returns a simple block associated with the current input token.

5.4.7. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-a-simple-block

# File lib/crass/parser.rb, line 460
def consume_simple_block(input = @tokens)
  @depth += 1

  begin
    # Discard blocks nested more deeply than the maximum allowed depth to
    # avoid exhausting the Ruby stack on maliciously nested input.
    return discard_block(input) if @depth > @maximum_depth

    start_token = input.current[:node]
    end_token   = BLOCK_END_TOKENS[start_token]

    block = {
      :start  => start_token.to_s,
      :end    => end_token.to_s,
      :value  => [],
      :tokens => [input.current] # Non-standard. Used for serialization.
    }

    block[:tokens].concat(input.collect do
      while token = input.consume
        break if token[:node] == end_token

        input.reconsume
        block[:value] << consume_component_value(input)
      end
    end)

    create_node(:simple_block, block)
  ensure
    @depth -= 1
  end
end
create_node(type, properties = {}) click to toggle source

Creates and returns a new parse node with the given properties.

# File lib/crass/parser.rb, line 526
def create_node(type, properties = {})
  {:node => type}.merge!(properties)
end
create_selector(input) click to toggle source

Parses the given input tokens into a selector node and returns it.

Doesn’t bother splitting the selector list into individual selectors or validating them. Feel free to do that yourself! It’ll be fun!

# File lib/crass/parser.rb, line 534
def create_selector(input)
  create_node(:selector,
    :value  => parse_value(input),
    :tokens => input)
end
create_style_rule(rule) click to toggle source

Creates a ‘:style_rule` node from the given qualified rule, and returns it.

# File lib/crass/parser.rb, line 542
def create_style_rule(rule)
  create_node(:style_rule,
    :selector => create_selector(rule[:prelude]),
    :children => parse_properties(rule[:block]))
end
discard_block(input) click to toggle source

Discards an over-nested simple block or function without recursing, then returns an ‘:error` node. Assumes `input.current` is the opening token (a `{`, `[`, `(`, or function token).

This is reached only when the configured maximum nesting depth is exceeded. It iteratively consumes tokens up to the matching closing token (tracking nested blocks and functions with an explicit stack) so that a deeply nested construct can’t exhaust the Ruby stack.

# File lib/crass/parser.rb, line 501
def discard_block(input)
  opening = input.current

  stack = [opening[:node] == :function ? :')' : BLOCK_END_TOKENS[opening[:node]]]
  tokens = [opening] # Non-standard. Used for serialization.

  tokens.concat(input.collect do
    until stack.empty?
      break unless token = input.consume

      case token[:node]
      when :'{', :'[', :'('
        stack.push(BLOCK_END_TOKENS[token[:node]])
      when :function
        stack.push(:')')
      when stack.last
        stack.pop
      end
    end
  end)

  create_node(:error, :value => 'maximum-depth-exceeded', :tokens => tokens)
end
parse_component_value(input = @tokens) click to toggle source

Parses a single component value and returns it.

5.3.7. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#parse-a-component-value

# File lib/crass/parser.rb, line 551
def parse_component_value(input = @tokens)
  input = TokenScanner.new(input) unless input.is_a?(TokenScanner)

  while input.peek && input.peek[:node] == :whitespace
    input.consume
  end

  if input.peek.nil?
    return create_node(:error, :value => 'empty')
  end

  value = consume_component_value(input)

  while input.peek && input.peek[:node] == :whitespace
    input.consume
  end

  if input.peek.nil?
    value
  else
    create_node(:error, :value => 'extra-input')
  end
end
parse_component_values(input = @tokens) click to toggle source

Parses a list of component values and returns an array of parsed tokens.

5.3.8. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#parse-a-list-of-component-values

# File lib/crass/parser.rb, line 578
def parse_component_values(input = @tokens)
  input  = TokenScanner.new(input) unless input.is_a?(TokenScanner)
  tokens = []

  while token = consume_component_value(input)
    tokens << token
  end

  tokens
end
parse_declaration(input = @tokens) click to toggle source

Parses a single declaration and returns it.

5.3.5. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#parse-a-declaration

# File lib/crass/parser.rb, line 592
def parse_declaration(input = @tokens)
  input = TokenScanner.new(input) unless input.is_a?(TokenScanner)

  while input.peek && input.peek[:node] == :whitespace
    input.consume
  end

  if input.peek.nil?
    # Syntax error.
    return create_node(:error, :value => 'empty')
  elsif input.peek[:node] != :ident
    # Syntax error.
    return create_node(:error, :value => 'invalid')
  end

  if decl = consume_declaration(input)
    return decl
  end

  # Syntax error.
  create_node(:error, :value => 'invalid')
end
parse_declarations(input = @tokens, options = {}) click to toggle source

Parses a list of declarations and returns them.

See {#consume_declarations} for options.

5.3.6. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#parse-a-list-of-declarations

# File lib/crass/parser.rb, line 620
def parse_declarations(input = @tokens, options = {})
  input = TokenScanner.new(input) unless input.is_a?(TokenScanner)
  consume_declarations(input, options)
end
parse_properties(input = @tokens) click to toggle source

Parses a list of declarations and returns an array of ‘:property` nodes (and any non-declaration nodes that were in the input). This is useful for parsing the contents of an HTML element’s ‘style` attribute.

# File lib/crass/parser.rb, line 628
def parse_properties(input = @tokens)
  properties = []

  parse_declarations(input).each do |decl|
    unless decl[:node] == :declaration
      properties << decl
      next
    end

    children = decl[:value].dup
    children.pop if children.last && children.last[:node] == :semicolon

    properties << create_node(:property,
      :name      => decl[:name],
      :value     => parse_value(decl[:value]),
      :children  => children,
      :important => decl[:important],
      :tokens    => decl[:tokens])
  end

  properties
end
parse_rule(input = @tokens) click to toggle source

Parses a single rule and returns it.

5.3.4. www.w3.org/TR/2013/WD-css-syntax-3-20130919/#parse-a-rule

# File lib/crass/parser.rb, line 654
def parse_rule(input = @tokens)
  input = TokenScanner.new(input) unless input.is_a?(TokenScanner)

  while input.peek && input.peek[:node] == :whitespace
    input.consume
  end

  if input.peek.nil?
    # Syntax error.
    return create_node(:error, :value => 'empty')
  elsif input.peek[:node] == :at_keyword
    rule = consume_at_rule(input)
  else
    rule = consume_qualified_rule(input)
  end

  while input.peek && input.peek[:node] == :whitespace
    input.consume
  end

  if input.peek.nil?
    rule
  else
    # Syntax error.
    create_node(:error, :value => 'extra-input')
  end
end
parse_value(nodes) click to toggle source

Returns the unescaped value of a selector name or property declaration.

# File lib/crass/parser.rb, line 683
def parse_value(nodes)
  nodes  = [nodes] unless nodes.is_a?(Array)
  string = String.new

  nodes.each do |node|
    case node[:node]
    when :comment, :semicolon
      next

    when :at_keyword, :ident
      string << node[:value]

    when :function
      if node[:value].is_a?(String)
        string << node[:value]
        string << '('
      else
        string << parse_value(node[:tokens])
      end

    else
      if node.key?(:raw)
        string << node[:raw]
      elsif node.key?(:tokens)
        string << parse_value(node[:tokens])
      end
    end
  end

  string.strip
end