53 Commits

Author SHA1 Message Date
Andreas Dangel
3a7460de29 [lang-test] Move classes into n.s.p.lang.test 2024-02-23 17:46:23 +01:00
Andreas Dangel
6163f67b06 [core] Rename Tokenizer to CpdLexer
See 
2024-01-11 17:04:48 +01:00
Andreas Dangel
32afa53f8e All language modules: getInstance()
Use a consistent implementation of getInstance().
Now all modules resolve against the LanguageRegistry.
2023-08-24 16:01:27 +02:00
Clément Fournier
629e3b415c Consolidate CPD packages 2023-06-10 14:30:07 +02:00
Clément Fournier
0f17cc83f6 Add back default version for CPD languages 2023-03-11 17:17:43 +01:00
Clément Fournier
c572cb88d7 Rename package cpd.internal to cpd.impl 2023-02-13 15:18:37 +01:00
Clément Fournier
ddbfc90c14 Fix build 2023-02-12 20:06:28 +01:00
Clément Fournier
8fbd830daa Style and renamings 2023-02-12 17:24:02 +01:00
Clément Fournier
cf81809990 Change a ton of stuff in CPD 2023-02-10 18:21:23 +01:00
Clément Fournier
d4c05d1fb5 Make pmd language have a hook to launch CPD
This allows removing the CPD language instances,
sharing more logic between CPD and PMD, and using
language properties to configure CPD and PMD uniformly.
2023-02-10 16:45:52 +01:00
Andreas Dangel
7ef6198a3b Merge branch 'pmd/7.0.x' into pmd7-junit5-part2 2022-07-22 21:22:45 +02:00
Andreas Dangel
378c77f60f [dart] Migrate tests to Junit5 2022-07-07 17:19:12 +02:00
Clément Fournier
efcfb665ad Cleanups for antlr tokenizers 2020-09-17 22:37:10 +02:00
Andreas Dangel
47ab1f34a4 Delete old antlr4 package-infos 2020-09-13 11:41:26 +02:00
Andreas Dangel
4732164a45 Merge branch 'pmd/7.0.x' into pr-2753 2020-09-13 11:40:23 +02:00
Andreas Dangel
3caa6dd0ba Merge branch 'master' into pmd/7.0.x 2020-09-13 11:39:37 +02:00
Andreas Dangel
7a1e3caba0 Add deprecations for antlr lexers that will be moved with PMD 7
Refs 
2020-09-13 11:28:07 +02:00
Clément Fournier
40443326d3 Add cleanup phase for antlr CPD languages
This removes the parser and other things we don't need in the API

Currently only pmd-dart has been adapted
2020-08-30 18:50:01 +02:00
Clément Fournier
ae26c29542 Merge branch 'master' into 7.0.x 2020-08-30 17:16:28 +02:00
Clément Fournier
13146b5c53 Merge branch 'master' into cpd_dart_escaped_dollar 2020-08-30 16:52:55 +02:00
Clément Fournier
e95c90a05a Merge branch 'master' into cpd_dart_string_interpolation 2020-08-30 16:47:38 +02:00
Clément Fournier
c2069282a7 Add test case for more complicated interpolations 2020-08-30 16:47:25 +02:00
Clément Fournier
7384fd6fb2 Merge branch 'master' into 7.0.x 2020-08-28 14:58:10 +02:00
Maikel Steneker
7740e47292 Escaped characters in Dart multiline string are now handled correctly 2020-08-27 14:21:10 +02:00
Maikel Steneker
74a9950128 Improvements for Dart interpolated strings
The Dart tokenizer was incomplete, resulting in some strings that use
string interpolation not being parsed correctly. This has been worked
around by making sure that the contents of string interpolation are all
included in the same string token.
2020-08-27 13:16:56 +02:00
XenoAmess
244a39db0f fix typos. 2020-08-27 11:47:11 +08:00
Andreas Dangel
2b1d071930 Fix tests due to endColumn changes 2020-08-21 12:38:37 +02:00
Andreas Dangel
ab489832fe Merge branch 'master' into pmd/7.0.x 2020-08-21 11:51:54 +02:00
Maikel Steneker
6fb5ac59b9 Ensure CPD uses tab width of 1 for tabs consistently
The columns that are reported by CPD were inconsistent across languages
before. A language like Java (using a JavaCC-based tokenizer) would use
a width of 8 for tabs, whereas a language like C# (using an Antlr-based
tokenizer) would use 1 instead.

This includes unit tests for most languages to ensure a tab character is
counted as 1. The configuration for JavaCC has been adjusted to respect
this as well.
2020-07-20 10:42:21 +02:00
Clément Fournier
5b7ed588ea Make CPD end column exclusive
This is to align it with PMD columns,
which in 7.0 are already exclusive.
2020-06-19 02:21:45 +02:00
Clément Fournier
c49d9e315f Merge branch 'master' into 7.0.x 2020-06-19 01:32:32 +02:00
Clément Fournier
551ab453fd Convert CPP tests 2020-06-13 20:14:08 +02:00
Clément Fournier
0a1e82efad Rename dir 2020-06-13 19:21:27 +02:00
Clément Fournier
43412b532b Fix column problem with Antlr multiline tokens 2020-06-12 18:15:33 +02:00
Clément Fournier
1540ec6d91 Port remaining dart tests 2020-06-12 16:56:43 +02:00
Clément Fournier
ecc8699269 Add doc 2020-06-12 16:27:40 +02:00
Clément Fournier
b47001c90d Add text comparison tests for CPD 2020-06-12 16:01:00 +02:00
Andreas Dangel
0710877d05 [core] Deprecate AntlrToken#getType() in favor of #getKind()
Refs 
2020-04-10 18:50:58 +02:00
Clément Fournier
2ea78db0dc Checkstyle 2020-03-22 00:56:10 +01:00
Clément Fournier
16f30c9ece Make Antlr token manager throw TokenMgrError directly 2020-03-22 00:48:16 +01:00
Clément Fournier
b51c02f93b Reorganise antlr files
Antlr root nodes need to implement RootNode.

Also, I didn't like the fact that the swift
module didn't use the conventional package
structure (lang.swift.ast) but was rolling
its own convention (lang.swift.antlr4).

I moved base classes for the antlr implementations
into nspmd.lang.ast.impl.antlr4. The fact that a
module is implemented with antlr is an implementation
detail, and it doesn't deserve its own toplevel package
in nspmd.lang.
2019-09-25 03:49:36 +02:00
Andreas Dangel
a1a13b6234 [dart] Add multi-line string example 2019-05-25 19:18:09 +02:00
Maikel Steneker
806b130e4a Improved Dart tokenizer to handle strings correctly.
This is based on the grammar at https://github.com/chalin/dart-spec-and-grammar/blob/master/doc/grammar-AUTOGENERATED-DO-NOT-EDIT.txt.

The string handling is now much closer to this grammar, resulting in more files being tokenized correctly.
2019-05-21 14:43:34 +02:00
Maikel Steneker
bcf0de6880 Various unit tests for strings that are currently not handled correctly by the Dart tokenizer. 2019-05-21 14:43:16 +02:00
Maikel Steneker
1ebfb7df87 Extended Dart grammar to take escape sequences into account. 2019-04-30 14:01:07 +02:00
Maikel Steneker
6f442b15ca Added Dart tokenizer test cases for escaped backslashes (\\) and other characters (\n, \$). 2019-04-30 13:58:12 +02:00
Maikel Steneker
26a64be1d2 Dart grammar now supports string containing escaped quotes. 2019-04-24 15:52:22 +02:00
Maikel Steneker
1e2e591ac4 Added unit test for string containing escaped quotes. 2019-04-24 15:43:53 +02:00
Juan Martín Sotuyo Dodero
ee227019aa Update changelog, refs 2019-04-10 16:17:25 -03:00
Maikel Steneker
7b78ac75d0 Minor formatting changes. 2019-04-05 17:22:31 +02:00