Code analyzer

code analyzer tool which is extracted from rails_best_practices

This project is maintained by flyerhzm

Introduction

code_analyzer gem is extracted from rails_best_practices, its aim is to provide a simple way to build your own ruby code analyzer tool.

Quick Start

1. create ruby file you want to analyze.

Let's create a simple ruby file test_class.rb, its content is as follow.

class TestClass
  # bad
  def TestClass.some_method
    # body omitted
  end

  # good
  def self.some_other_method
    # body omitted
  end
end

The code is picked from github ruby styleguide, let's write a code analyzer script to detect bad style. TestClass.some_method

2. understand the corresponding sexp you need to analyze.

code_analyzer is analyzing sexp which is generated by ruby ripper, you should have a basic idea of what sexp is. I have built a website try-ripper, which helps you easily understand sexp. Copy the source code of test_class.rb to "Ruby Code" textarea, and click "Convert" button, then you will see the corresponding Ripper Result. The sexp for TestClass.some_method is

s(:var_ref, s(:@const, "TestClass", s(1, 4)))

and sexp for self.some_method is

s(:var_ref, s(:@kw, "self", s(2, 4)))

Be aware I only paste the difference here. As you can see, what we only need to do is to check all class method definitions (defs), and check if the first child is var_ref with @const, if so, it is a bad style.

3. add code_analyzer dependency

add Gemfile

source :rubygems

gem "code_analyzer"

and run bundle install.

4. write a checker to detect bad style.

code_analyzer allows you easily write your own ruby code checker.

class ClassMethodChecker < CodeAnalyzer::Checker
  interesting_files /.*\.rb/
  interesting_nodes :defs

  add_callback :start_defs do |node|
    if :var_ref == node[1].sexp_type && :@const == node[1][1].sexp_type
      add_warning "use self to define class method"
    end
  end
end

It said ClassMethodChecker will check all ruby files, and check only class method definition, if it's first child (without sexp_type) is var_ref with @const, add warning to say "use self to define class method".

5. analyze ruby code with code_analyzer

Now we are ready to analyze test_class.rb code.

require 'code_analyzer'
require './class_method_checker'

filename = "test_class.rb"
content = File.read(filename)

checker = ClassMethodChecker.new
visitor = CodeAnalyzer::CheckingVisitor::Default.new(checkers: [checker])
visitor.check(filename, content)
visitor.warnings.each { |warning| puts warning }

CodeAnalyzer::CheckingVisitor::Default is used to visit all sexp nodes, here we register ClassMethodChecker in visitor, and check the test_class.rb, then we will see the following output.

test_class.rb:3 - use self to define class method

Okay, we have finished a simple ruby code analyzer tool. You can get the sample code here.

Document

Checker

Checker is responsible for checking ruby code to see if it is good for you. Writing a custom checker is easier than you expected, you only need to tell checker what files you want to check, what sexp nodes you want to check and how to check them. That's it, let's see how to do these 3 steps.

1. tell what files you want to check.

class MyChecker < CodeAnalyzer::Checker
  interesting_files %r|app/models/.*\.rb|, %r|lib/.*\.rb|
end

interesting_files accepts one or more regular expressions, the checker only parses files whose filename match any of the given regular expression. Here the MyChecker only checks ruby code under app/modes and lib directories.

2. tell what sexp node you want to check.

class MyChecker < CodeAnalyzer::Checker
  interesting_files %r|app/models/.*\.rb|, %r|lib/.*\.rb|
  interesting_nodes :def, :defs
end

interesting_nodes accepts one or more sexp types, the checker only cares about the given sexp nodes. Here the MyChecker only checks def (instance method definition) and defs (class method definition) nodes.

3. how to check.

class MyChecker < CodeAnalyzer::Checker
  interesting_files %r|app/models/.*\.rb|, %r|lib/.*\.rb|
  interesting_nodes :def, :defs

  add_callback :start_def do |node|
    # this node is a :def node, analyzing it here.
  end

  add_callback :start_defs do |node|
    # this node is a defs node, analyzing it here.
  end
end

add_callback allows you define callbacks before or after visiting a sexp node. The first argument is the callback name, like start_def or end_def, the second is a block with sexp node instance, in the block, you can analyze the sexp node, like if it violates any code guideline in your team.

There is a special callback, named after_check, which won't be triggered during sexp nodes visit, it will be called by CheckingVisitor manually.

CheckingVisitor

CheckingVisitor will visit all ruby sexp nodes, triggering the callbacks based on the current sexp node.

1. we should tell CheckingVisitor what checkers we need.

checkers = [MyChecker.new, YourChecker.new]
visitor = CodeAnalyzer::CheckingVisitor::Default.new(checkers: checkers)

Here CheckingVisitor registers MyChecker and YourChecker's callbacks.

2. visiting all sexp nodes.

filename = "app/models/user.rb"
visitor.check(filename, File.read(filename))

check accepts 2 arguments, one is the filename, the other is file content. The reason it needs both filename and file content as arguments is that it allows you do some conversion, like ignoring html tags in erb before checking. check method will visit all sexp nodes and trigger the registered callbacks.

3. trigger after_check if necessary.

visitor.after_check

it will trigger all after_check callbacks.

Warning

Warning wraps warning message, filename and line number, so you can know what code should be refactored after analyzing.

1. add warning during analyzing. 

class MyChecker < CodeAnalyzer::Checker
  ...
  add_callback :start_def do |node|
    # analyze it
    add_warning "it violates blah blah"
  end
end

add_warning not only records the warning message you assign, but also records the current filename and current line number automatically.

2. read warnings after analyzing.

checkers = [MyChecker.new]
visitor = CodeAnalyzer::CheckingVisitor::Default.new(checkers: checkers)
visitor.check(filename, File.read(filename))
visitor.warnings.each { |warning| puts warning }

CheckingVisitor's warnings will return warnings in all registered checkers.

Sexp

code_analyzer adds a lot of extensions for sexp, e.g.

Sexp#line_number # return the line number of current sexp node.
Sexp#grep_nodes(options) # return all matching sexp nodes.
Sexp#grep_node(options) # return the first matching sexp node.
Sexp#class_name # return the class name for class node.
Sexp#method_name # return the method name for def or defs node.
Sexp#receiver # return the receiver for call node.
Sexp#message # return the message for call node
Sexp#arguments # return the arguments for def or defs node
...
To get full api list, please check out here.

Use Cases

Help

If you have any questions or suggestions, feel free to contact me