codemod 0.1.5

codemod for Dart #

Pub Build Status

A library that makes it easy to write and run automated code modifications on a codebase. Primarily geared towards updating/refactoring Dart code by leveraging the analyzer package's APIs for parsing and traversing the AST.

Inspired by and based on Facebook's codemod library.

Demo #

demo

How It Works #

The end goal of this library is to enable you to easily and automatically apply code modifications and refactors via an interactive CLI. To that end, the following function is provided:

int runInteractiveCodemod(FileQuery query, Suggestor suggestor);

Calling this will tell codemod to scan for files using query, which will return a set of file paths that codemod will then read. Each file is then provided as the input to the suggestor, which will return a list of patches that should be suggested to the user. As patches are suggested and accepted by the user, codemod handles applying them to the files and writing the result to disk.

Writing a Suggestor #

This library provides Suggestor, but it is just an interface with two methods:

abstract class Suggestor {
  bool shouldSkip(String sourceFileContents);
  Iterable<Patch> generatePatches(SourceFile sourceFile);
}

Codemod will read the contents of each file returned from the query and first pass it to shouldSkip(). This provides a way to short-circuit the potentially expensive generatePatches() method if need be.

If not skipped, the file contents will be passed to generatePatches() in the form of a SourceFile from the source_span package. Operating on this model makes it easy to create patches at specific offsets within the file.

Suggestor Example: Insert License Headers #

The following suggestor checks each file for the expected license header, and if missing, yields a Patch that inserts it at the beginning of the file.

import 'package:codemod/codemod.dart';
import 'package:source_span/source_span.dart';

final String licenseHeader = '''
// Lorem ispum license.
// 2018-2019
''';

class LicenseHeaderInserter implements Suggestor {
  @override
  bool shouldSkip(String sourceFileContents) =>
      sourceFileContents.trimLeft().startsWith(licenseHeader);

  @override
  Iterable<Patch> generatePatches(SourceFile sourceFile) sync* {
    yield Patch(
      sourceFile,
      // The span across which the patch should be applied.
      sourceFile.span(
        // Start offset.
        // 0 means "insert at the beginning of the file."
        0,
        // End offset.
        // Using the same offset as the start offset here means that the patch
        // is being inserted at this point instead of replacing a span of text.
        0,
      ),
      // Text to insert.
      licenseHeader,
    );
  }
}

Suggestor Example: Regex Substitution #

Regex substitutions are also a common strategy for codemods and are sufficient for simple changes. The following suggestor updates a version constraint for the codemod package in a pubspec.yaml:

import 'package:codemod/codemod.dart';
import 'package:source_span/source_span.dart';

/// Pattern that matches a dependency version constraint line for the `codemod`
/// package, with the first capture group being the constraint.
final RegExp pattern = RegExp(
  r'''^\s*codemod:\s*([\d\s"'<>=^.]+)\s*$''',
  multiLine: true,
);

/// The version constraint that `codemod` entries should be updated to.
const String targetConstraint = '^1.0.0';

class RegexSubstituter implements Suggestor {
  @override
  bool shouldSkip(String sourceFileContents) => false;

  @override
  Iterable<Patch> generatePatches(SourceFile sourceFile) sync* {
    final contents = sourceFile.getText(0);
    for (final match in pattern.allMatches(contents)) {
      final line = match.group(0);
      final constraint = match.group(1);
      final updated = line.replaceFirst(constraint, targetConstraint) + '\n';

      yield Patch(
        sourceFile,
        sourceFile.span(match.start, match.end),
        updated,
      );
    }
  }
}

Suggestor Example: AST Visitor #

Regexes and custom parsing can get you pretty far, but using the analyzer's visitor pattern to traverse the parsed AST is a much more robust option and allows for the creation of very powerful codemods with relatively little effort.

Consider the following suggestor that removes all deprecated declarations (i.e. classes, constructors, variables, methods, etc.):

import 'package:analyzer/analyzer.dart';
import 'package:codemod/codemod.dart';

class DeprecatedRemover extends GeneralizingAstVisitor
    with AstVisitingSuggestorMixin {
  static bool isDeprecated(AnnotatedNode node) =>
      node.metadata.any((m) => m.name.name.toLowerCase() == 'deprecated');

  @override
  visitDeclaration(Declaration node) {
    if (isDeprecated(node)) {
      // Remove the node by replacing the span from its start offset to its end
      // offset with an empty string.
      yieldPatch(node.offset, node.end, '');
    }
  }
}

In this example, the suggestor extends the GeneralizingAstVisitor which allows it to target all nodes that could be deprecated with a single visit method. Then it's just a matter of checking for either the @Deprecated() or @deprecated() annotation and yielding a patch with an empty string across the entire node, which is effectively a deletion.

You may notice that in this example, the suggestor is no longer implementing generatePatches() – instead, we use the AstVisitingSuggestorMixin. This mixin handles parsing the AST for the given SourceFile and starting the visitor pattern so that all you have to do is override the applicable visit methods.

Additionally, although the GeneralizingAstVisitor was the appropriate choice for this suggestor, any AstVisitor will work. Choose whichever one fits the job.

If you're not familiar with the analyzer API, in particular the AstNode class hierarchy and the AstVisitor pattern, it may be a good opportunity to browse the analyzer source code or look at the AST visiting suggestor codemods that are linked below in the references section to see what is possible with this approach.

Running a Codemod #

All you need to run a codemod is:

  1. A FileQuery to determine the set of files to be read

  2. A Suggestor to suggest patches on each file

  3. A .dart file with a main() block that calls runInterativeCodemod()

If we were to run the 3 suggestor examples from above, it would like like so:

Regex Substituter:

import 'dart:io';
import 'package:codemod/codemod.dart';

void main(List<String> args) {
  exitCode = runInteractiveCodemod(
    FileQuery.single('pubspec.yaml'),
    RegexSubstituter(),
    args: args,
  );
}

License Header Inserter:

import 'dart:io';
import 'package:codemod/codemod.dart';

void main(List<String> args) {
  exitCode = runInteractiveCodemod(
    FileQuery.dir(pathFilter: isDartFile),
    LicenseHeaderInserter(),
    args: args,
  );
}

Deprecated Remover:

import 'dart:io';
import 'package:codemod/codemod.dart';

void main(List<String> args) {
  exitCode = runInteractiveCodemod(
    FileQuery.dir(pathFilter: isDartFile),
    DeprecatedRemover(),
    args: args,
  );
}

Run the .dart file directly or package it up as an executable and publish it on pub!

Additional Options #

To facilitate the creation of more complex codemods, two additional pieces are provided by this library:

  • Aggregate multiple suggestors into a single suggestor with AggregateSuggestor:

      import 'dart:io';
      import 'package:codemod/codemod.dart';
    
      void main(List<String> args) {
        final query = ...;
        exitCode = runInteractiveCodemod(
          query,
          AggregateSuggestor([
            SuggestorA(),
            SuggestorB(),
          ]),
        );
      }
    
  • Run multiple suggestors (or aggregate suggestors) sequentially:

      import 'dart:io';
      import 'package:codemod/codemod.dart';
    
      void main(List<String> args) {
        final query = ...;
        exitCode = runInteractiveCodemodSequence(
          query,
          [
            PhaseOneSuggestor(),
            PhaseTwoSuggestor(),
          ],
          args: args,
        );
      }
    

    This can be useful if a certain modification needs to happen prior to another, or if you need to use a "collector" pattern wherein the first suggestor collects information from the files that a second suggestor will then use to suggest patches.

Testing Suggestors #

Testing suggestors is relatively easy for two reasons:

  • The API surface area is small (most of the time you only need to test the generatePatches() method)

  • The list of patches returned by generatePatches() can be applied to the input SourceFile to obtain a String output, which is trivial to examine in order to assert correctness.

In other words, all you need to do is determine a sufficient set of inputs and their respective expected outputs.

To help out, this library exports the applyPatches(sourceFile, patches) function that it uses internally to make it easy to compare the result of a suggestor's patches to the expected output.

Let's use the DeprecatedRemover suggestor example from above to demonstrate testing:

import 'package:codemod/codemod.dart';
import 'package:source_span/source_span.dart';
import 'package:test/test.dart';

void main() {
  group('DeprecatedRemover', () {
    test('removes deprecated variable', () {
      final sourceFile = SourceFile.fromString('''
// Not deprecated.
var foo = 'foo';
@deprecated
var bar = 'bar';''');
      final expectedOutput = '''
// Not deprecated.
var foo = 'foo';
''';

      final patches = DeprecatedRemover().generatePatches(sourceFile);
      expect(patches, hasLength(1));
      expect(applyPatches(sourceFile, patches), expectedOutput);
    });
  });
}

References #

Credits #


Contributing #

  • Run tests: pub run test

  • Format code: pub run dart_dev format

  • Run static analysis: dartanalyzer .

0.1.4 #

  • Prompts the user to either skip overlapping patches or quit when they are found.

0.1.3 #

  • Codemod authors can now augment the help output and the changes required output via runInteractiveCodemod() and runInteractiveCodemodSequence() using the optional additionalHelpOutput and changesRequiredOutput params.

    • If additionalHelpOutput is given, it will be printed to stderr after the default help output when the codemod is run with the -h|--help flag.

    • If changesRequiredOutput is given, it will be printed to stderr after the default output when the codemod is run with the --fail-on-changes flag and changes are in fact required.

0.1.2 #

  • Fix a typing issue with the AggregateSuggestor's constructor param.

  • Add tests for AggregateSuggestor and AstVisitingSuggestorMixin

0.1.1 #

  • Update pubspec.yaml for initial OSS release.

0.1.0 #

  • Initial tag.

Use this package as a library

1. Depend on it

Add this to your package's pubspec.yaml file:


dependencies:
  codemod: ^0.1.5

2. Install it

You can install packages from the command line:

with pub:


$ pub get

with Flutter:


$ flutter pub get

Alternatively, your editor might support pub get or flutter pub get. Check the docs for your editor to learn more.

3. Import it

Now in your Dart code, you can use:


import 'package:codemod/codemod.dart';
  
Popularity:
Describes how popular the package is relative to other packages. [more]
53
Health:
Code health derived from static analysis. [more]
99
Maintenance:
Reflects how tidy and up-to-date the package is. [more]
100
Overall:
Weighted score of the above. [more]
76
Learn more about scoring.

We analyzed this package on Aug 21, 2019, and provided a score, details, and suggestions below. Analysis was completed with status completed using:

  • Dart: 2.4.0
  • pana: 0.12.19

Platforms

Detected platforms: Flutter, other

Primary library: package:codemod/codemod.dart with components: io.

Health suggestions

Fix lib/src/suggestors.dart. (-1 points)

Analysis of lib/src/suggestors.dart reported 2 hints:

line 15 col 1: 'package:analyzer/analyzer.dart' is deprecated and shouldn't be used.

line 190 col 29: 'parseCompilationUnit' is deprecated and shouldn't be used.

Maintenance suggestions

Maintain an example.

None of the files in the package's example/ directory matches known example patterns.

Common filename patterns include main.dart, example.dart, and codemod.dart. Packages with multiple examples should provide example/README.md.

For more information see the pub package layout conventions.

Dependencies

Package Constraint Resolved Available
Direct dependencies
Dart SDK >=2.2.1 <3.0.0
analyzer >=0.37.0 <0.39.0 0.38.1
args ^1.5.1 1.5.2
io ^0.3.3 0.3.3
logging ^0.11.3+2 0.11.3+2
meta ^1.1.7 1.1.7
path ^1.6.2 1.6.4
source_span ^1.4.1 1.5.5
stack_trace ^1.9.3 1.9.3
Transitive dependencies
async 2.3.0
charcode 1.1.2
collection 1.14.12
convert 2.1.1
crypto 2.1.2
csslib 0.16.1
front_end 0.1.23
glob 1.1.7
html 0.14.0+2
kernel 0.3.23
package_config 1.1.0
pub_semver 1.4.2
string_scanner 1.0.5
term_glyph 1.1.0
typed_data 1.1.6
watcher 0.9.7+12
yaml 2.1.16
Dev dependencies
build_runner ^1.0.0
build_test ^0.10.8
dart_dev ^2.0.1
dart_style ^1.2.0
dependency_validator ^1.2.3
mockito ^4.0.0
pedantic ^1.4.0 1.8.0+1
test ^1.5.1+1