hiifong/pmd
1
0
Fork 0
forked from phoedos/pmd
Code Pull Requests Activity

Merge pull request #4726 from Monits:upgrade-groovy

Browse Source 
[groovy] Support Groovy to 3 and 4 and CPD suppressions #4726
This commit is contained in:
Andreas Dangel 2023-12-11 13:42:29 +01:00
commit e4f2307b06
No known key found for this signature in database
GPG Key ID: 93450DF2DF9A3FA3
13 changed files with 680 additions and 96 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
docs/pages/pmd/languages/groovy.md
View File

@ -11,3 +11,11 @@ summary: "Groovy-specific features and guidance"
> familiar and easy to learn syntax.
{% include language_info.html name='Groovy' id='groovy' implementation='groovy::lang.groovy.GroovyLanguageModule' supports_cpd=true since='5.5.2' %}
## Support in PMD
Groovy support was added with PMD 5.5.2. With PMD 7.0.0, support for Groovy 3 and 4 was added.
Since PMD 7.0.0, the Groovy module supports [suppression](pmd_userdocs_cpd.html#suppression) through `CPD-ON`/`CPD-OFF` comment pairs.
### Limitations
- Support for Groovy only extends to CPD to detect code duplication.

2
docs/pages/pmd/userdocs/cpd/cpd.md
View File

@ -477,7 +477,7 @@ Here's a screenshot of CPD after running on the JDK 8 java.lang package:
## Suppression
Arbitrary blocks of code can be ignored through comments on **Java**, **C/C++**, **Dart**, **Go**, **Javascript**,
Arbitrary blocks of code can be ignored through comments on **Java**, **C/C++**, **Dart**, **Go**, **Groovy**, **Javascript**,
**Kotlin**, **Lua**, **Matlab**, **Objective-C**, **PL/SQL**, **Python**, **Scala**, **Swift** and **C#** by including the keywords `CPD-OFF` and `CPD-ON`.
```java

18
docs/pages/release_notes.md
View File

@ -77,6 +77,12 @@ in the Migration Guide.
* limited support for Swift 5.9 (Macro Expansions)
##### Groovy Support (CPD)
* We now support parsing all Groovy features from Groovy 3 and 4.
* We now support [suppression](pmd_userdocs_cpd.html#suppression) through `CPD-ON`/`CPD-OFF` comment pairs.
* See [PR #4726](https://github.com/pmd/pmd/pull/4726) for details.
#### Rule Changes
**New Rules**
@ -99,6 +105,7 @@ in the Migration Guide.
* [#4723](https://github.com/pmd/pmd/issues/4723): \[cli] Launch fails for "bash pmd"
* core
* [#1027](https://github.com/pmd/pmd/issues/1027): \[core] Apply the new PropertyDescriptor<Pattern> type where applicable
* [#4674](https://github.com/pmd/pmd/issues/4674): \[core] WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass
* [#4750](https://github.com/pmd/pmd/pull/4750): \[core] Fix flaky SummaryHTMLRenderer
* doc
* [#3175](https://github.com/pmd/pmd/issues/3175): \[doc] Document language module features
@ -113,6 +120,8 @@ in the Migration Guide.
* [#4749](https://github.com/pmd/pmd/pull/4749): Fixes NoSuchMethodError on processing errors in pmd-compat6
* apex-performance
* [#4675](https://github.com/pmd/pmd/issues/4675): \[apex] New Rule: OperationWithHighCostInLoop
* groovy
* [#4726](https://github.com/pmd/pmd/pull/4726): \[groovy] Support Groovy to 3 and 4 and CPD suppressions
* java
* [#4753](https://github.com/pmd/pmd/issues/4753): \[java] PMD crashes while using generics and wildcards
* java-codestyle
@ -309,6 +318,12 @@ Note: Support for Java 19 preview language features have been removed. The versi
With the new version of Apex Jorje, the new language constructs like User Mode Database Operations
can be parsed now. PMD should now be able to parse Apex code up to version 59.0 (Winter '23).
#### Changed: Groovy Support (CPD)
* We now support parsing all Groovy features from Groovy 3 and 4.
* We now support [suppression](pmd_userdocs_cpd.html#suppression) through `CPD-ON`/`CPD-OFF` comment pairs.
* See [PR #4726](https://github.com/pmd/pmd/pull/4726) for details.
#### Changed: Rule properties
* The old deprecated classes like `IntProperty` and `StringProperty` have been removed. Please use
@ -555,6 +570,7 @@ See also [Detailed Release Notes for PMD 7]({{ baseurl }}pmd_release_notes_pmd7.
* [#4454](https://github.com/pmd/pmd/issues/4454): \[core] "Unknown option: '-min'" but is referenced in documentation
* [#4611](https://github.com/pmd/pmd/pull/4611): \[core] Fix loading language properties from env vars
* [#4621](https://github.com/pmd/pmd/issues/4621): \[core] Make `ClasspathClassLoader::getResource` child first
* [#4674](https://github.com/pmd/pmd/issues/4674): \[core] WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass
* [#4750](https://github.com/pmd/pmd/pull/4750): \[core] Fix flaky SummaryHTMLRenderer
* cli
* [#2234](https://github.com/pmd/pmd/issues/2234): \[core] Consolidate PMD CLI into a single command
@ -598,6 +614,8 @@ Language specific fixes:
* [#4675](https://github.com/pmd/pmd/issues/4675): \[apex] New Rule: OperationWithHighCostInLoop
* apex-security
* [#4646](https://github.com/pmd/pmd/issues/4646): \[apex] ApexSOQLInjection does not recognise SObjectType or SObjectField as safe variable types
* groovy
* [#4726](https://github.com/pmd/pmd/pull/4726): \[groovy] Support Groovy to 3 and 4 and CPD suppressions
* java
* [#520](https://github.com/pmd/pmd/issues/520): \[java] Allow `@SuppressWarnings` with constants instead of literals
* [#864](https://github.com/pmd/pmd/issues/864): \[java] Similar/duplicated implementations for determining FQCN

6
docs/pages/release_notes_pmd7.md
View File

@ -279,6 +279,12 @@ Related issue: [[core] Explicitly name all language versions (#4120)](https://gi
With the new version of Apex Jorje, the new language constructs like User Mode Database Operations
can be parsed now. PMD should now be able to parse Apex code up to version 59.0 (Winter '23).
### Changed: Groovy Support (CPD)
* We now support parsing all Groovy features from Groovy 3 and 4.
* We now support [suppression](pmd_userdocs_cpd.html#suppression) through `CPD-ON`/`CPD-OFF` comment pairs.
* See [PR #4726](https://github.com/pmd/pmd/pull/4726) for details.
## 🌟 New and changed rules
### New Rules

2
pmd-groovy/pom.xml
View File

@ -31,7 +31,7 @@
<artifactId>pmd-core</artifactId>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<groupId>org.apache.groovy</groupId>
<artifactId>groovy</artifactId>
</dependency>

93
pmd-groovy/src/main/java/net/sourceforge/pmd/lang/groovy/ast/impl/antlr4/GroovyToken.java Normal file
View File

@ -0,0 +1,93 @@
/*
* BSD-style license; for more info see http://pmd.sourceforge.net/license.html
*/
package net.sourceforge.pmd.lang.groovy.ast.impl.antlr4;
import net.sourceforge.pmd.annotation.Experimental;
import net.sourceforge.pmd.lang.ast.GenericToken;
import net.sourceforge.pmd.lang.ast.impl.antlr4.AntlrToken;
import net.sourceforge.pmd.lang.document.FileLocation;
import net.sourceforge.pmd.lang.document.TextDocument;
import net.sourceforge.pmd.lang.document.TextRegion;
import groovyjarjarantlr4.v4.runtime.Lexer;
import groovyjarjarantlr4.v4.runtime.Token;
/**
* A Groovy specific token representation.
*
* This is simply a copy of {@link AntlrToken} but
* referencing the jarjared version of antlr4 used by the groovy lexer.
*/
public class GroovyToken implements GenericToken<GroovyToken> {
private final Token token;
private final GroovyToken previousComment;
private final TextDocument textDoc;
GroovyToken next;
/**
* Constructor
*
* @param token The antlr token implementation
* @param previousComment The previous comment
* @param textDoc The text document
*/
public GroovyToken(final Token token, final GroovyToken previousComment, TextDocument textDoc) {
this.token = token;
this.previousComment = previousComment;
this.textDoc = textDoc;
}
@Override
public GroovyToken getNext() {
return next;
}
@Override
public GroovyToken getPreviousComment() {
return previousComment;
}
@Override
public CharSequence getImageCs() {
return token.getText();
}
/** Returns a text region with the coordinates of this token. */
@Override
public TextRegion getRegion() {
return TextRegion.fromBothOffsets(token.getStartIndex(), token.getStopIndex() + 1);
}
@Override
public FileLocation getReportLocation() {
return textDoc.toLocation(getRegion());
}
@Override
public boolean isEof() {
return getKind() == Token.EOF;
}
@Override
public int compareTo(GroovyToken o) {
return getRegion().compareTo(o.getRegion());
}
@Override
@Experimental
public int getKind() {
return token.getType();
}
public boolean isHidden() {
return !isDefault();
}
public boolean isDefault() {
return token.getChannel() == Lexer.DEFAULT_TOKEN_CHANNEL;
}
}

88
pmd-groovy/src/main/java/net/sourceforge/pmd/lang/groovy/ast/impl/antlr4/GroovyTokenManager.java Normal file
View File

@ -0,0 +1,88 @@
/*
* BSD-style license; for more info see http://pmd.sourceforge.net/license.html
*/
package net.sourceforge.pmd.lang.groovy.ast.impl.antlr4;
import org.apache.groovy.parser.antlr4.GroovyLexer;
import net.sourceforge.pmd.lang.TokenManager;
import net.sourceforge.pmd.lang.ast.TokenMgrError;
import net.sourceforge.pmd.lang.ast.impl.antlr4.AntlrTokenManager;
import net.sourceforge.pmd.lang.document.TextDocument;
import groovyjarjarantlr4.v4.runtime.ANTLRErrorListener;
import groovyjarjarantlr4.v4.runtime.Lexer;
import groovyjarjarantlr4.v4.runtime.RecognitionException;
import groovyjarjarantlr4.v4.runtime.Recognizer;
/**
* A Groovy specific token manager.
*
* This is simply a copy of {@link AntlrTokenManager} but
* referencing the jarjared version of antlr4 used by the groovy lexer.
*/
public class GroovyTokenManager implements TokenManager<GroovyToken> {
private final Lexer lexer;
private final TextDocument textDoc;
private GroovyToken previousToken;
public GroovyTokenManager(final Lexer lexer, final TextDocument textDocument) {
this.lexer = lexer;
this.textDoc = textDocument;
resetListeners();
}
@Override
public GroovyToken getNextToken() {
GroovyToken nextToken = getNextTokenFromAnyChannel();
while (!nextToken.isDefault()) {
nextToken = getNextTokenFromAnyChannel();
}
return nextToken;
}
private GroovyToken getNextTokenFromAnyChannel() {
/*
* Groovy's grammar doesn't hide away comments in a separate channel,
* but includes them as NL tokens with a different image
* See: https://github.com/apache/groovy/blob/GROOVY_4_0_15/src/antlr/GroovyLexer.g4#L980-L988
*/
final GroovyToken previousComment;
if (previousToken != null && previousToken.getKind() == GroovyLexer.NL
&& !"\n".equals(previousToken.getImage())) {
previousComment = previousToken;
} else {
previousComment = null;
}
final GroovyToken currentToken = new GroovyToken(lexer.nextToken(), previousComment, textDoc);
if (previousToken != null) {
previousToken.next = currentToken;
}
previousToken = currentToken;
return currentToken;
}
private void resetListeners() {
lexer.removeErrorListeners();
lexer.addErrorListener(new ErrorHandler());
}
private final class ErrorHandler implements ANTLRErrorListener<Object> {
@Override
public void syntaxError(final Recognizer recognizer,
final Object offendingSymbol,
final int line,
final int charPositionInLine,
final String msg,
final RecognitionException ex) {
throw new TokenMgrError(line, charPositionInLine, textDoc.getFileId(), msg, ex);
}
}
}

49
pmd-groovy/src/main/java/net/sourceforge/pmd/lang/groovy/cpd/GroovyTokenizer.java
View File

@ -4,50 +4,27 @@
package net.sourceforge.pmd.lang.groovy.cpd;
import org.codehaus.groovy.antlr.SourceInfo;
import org.codehaus.groovy.antlr.parser.GroovyLexer;
import java.io.IOException;
import net.sourceforge.pmd.cpd.TokenFactory;
import net.sourceforge.pmd.cpd.Tokenizer;
import org.apache.groovy.parser.antlr4.GroovyLexer;
import net.sourceforge.pmd.cpd.impl.TokenizerBase;
import net.sourceforge.pmd.lang.TokenManager;
import net.sourceforge.pmd.lang.document.TextDocument;
import net.sourceforge.pmd.lang.groovy.ast.impl.antlr4.GroovyToken;
import net.sourceforge.pmd.lang.groovy.ast.impl.antlr4.GroovyTokenManager;
import groovyjarjarantlr.Token;
import groovyjarjarantlr.TokenStream;
import groovyjarjarantlr.TokenStreamException;
import groovyjarjarantlr4.v4.runtime.CharStream;
import groovyjarjarantlr4.v4.runtime.CharStreams;
/**
* The Groovy Tokenizer
*/
public class GroovyTokenizer implements Tokenizer {
public class GroovyTokenizer extends TokenizerBase<GroovyToken> {
@Override
public void tokenize(TextDocument document, TokenFactory tokens) {
GroovyLexer lexer = new GroovyLexer(document.newReader());
TokenStream tokenStream = lexer.plumb();
try {
Token token = tokenStream.nextToken();
while (token.getType() != Token.EOF_TYPE) {
String tokenText = token.getText();
int lastCol;
int lastLine;
if (token instanceof SourceInfo) {
lastCol = ((SourceInfo) token).getColumnLast();
lastLine = ((SourceInfo) token).getLineLast();
} else {
// fallback
lastCol = token.getColumn() + tokenText.length();
lastLine = token.getLine(); // todo inaccurate
}
tokens.recordToken(tokenText, token.getLine(), token.getColumn(), lastLine, lastCol);
token = tokenStream.nextToken();
}
} catch (TokenStreamException err) {
throw tokens.makeLexException(lexer.getLine(), lexer.getColumn(), err.getMessage(), err);
}
protected final TokenManager<GroovyToken> makeLexerImpl(TextDocument doc) throws IOException {
CharStream charStream = CharStreams.fromReader(doc.newReader(), doc.getFileId().getAbsolutePath());
return new GroovyTokenManager(new GroovyLexer(charStream), doc);
}
}

6
pmd-groovy/src/test/java/net/sourceforge/pmd/lang/groovy/cpd/GroovyTokenizerTest.java
View File

@ -14,9 +14,13 @@ class GroovyTokenizerTest extends CpdTextComparisonTest {
super("groovy", ".groovy");
}
@Test
void testSample() {
doTest("sample");
}
@Test
void testCpdOffAndOn() {
doTest("cpdoff");
}
}

83
pmd-groovy/src/test/resources/net/sourceforge/pmd/lang/groovy/cpd/testdata/cpdoff.groovy vendored Normal file
View File

@ -0,0 +1,83 @@
// Copied from https://github.com/zeebo404/btree
package net.sourceforge.pmd.cpd
/**
* User: Eric
* Date: 4/30/2015
*/
class BTree<K, V> extends BTreeNode<K> {
static def instance
BlockManager<V> manager
BTree() {
instance = this
getLeaf(this)
manager = new BlockManager<>()
}
// CPD-OFF
def split() {
// create two new children
BTreeNode<K> left = clone()
BTreeNode<K> right = clone()
// assign parent to this
[left, right]*.parent = this
// Assign the left and right pointer lists
left.pointers = pointers.subList(0, count / 2 as int) as LinkedList
right.pointers = pointers.subList(count / 2 as int, count) as LinkedList
// clear the rightmost left key
if (left.internalNode) {
left.pointers[-1].key = null
}
else {
left.rightSibling = right
right.leftSibling = left
}
// reassign the parent node if not buckets
if (!bucketNode) {
[left, right].each { node -> node.pointers*.value*.parent = node }
}
// add the children to this
pointers.clear()
addDirect(new BTreeEntry(right.smallestKey, left))
addDirect(new BTreeEntry(null, right))
// Transform into a pointer node
if (leafNode) {
getPointer(this)
}
}
// CPD-ON
def add(K key, V value) {
if (count > 0 && search(key)) {
throw new IllegalArgumentException("$key is already in the tree")
}
BlockManager.Block.BlockElement<V> element = manager.element
element.value = value
super.add key, element
}
def delete(K key) {
if (count > 0 && !search(key)) {
throw new IllegalArgumentException("$key is not in the tree")
}
super.delete key
if (count == 0) {
getLeaf(this)
}
}
}

243
pmd-groovy/src/test/resources/net/sourceforge/pmd/lang/groovy/cpd/testdata/cpdoff.txt vendored Normal file
View File

@ -0,0 +1,243 @@
[Image] or [Truncated image[ Bcol Ecol
L1
[// Copied from https://github.com/[ 1 49
[\n] 49 50
L2
[package] 1 8
[net] 9 12
[.] 12 13
[sourceforge] 13 24
[.] 24 25
[pmd] 25 28
[.] 28 29
[cpd] 29 32
[\n] 32 33
L3
[/**\n * User: Eric\n * Date: 4/30/[ 1 4
L6
[\n] 4 5
L7
[class] 1 6
[BTree] 7 12
[<] 12 13
[K] 13 14
[,] 14 15
[V] 16 17
[>] 17 18
[extends] 19 26
[BTreeNode] 27 36
[<] 36 37
[K] 37 38
[>] 38 39
[{] 40 41
[\n] 41 42
L8
[\n] 1 2
L9
[static] 2 8
[def] 9 12
[instance] 13 21
[\n] 21 22
L10
[\n] 1 2
L11
[BlockManager] 2 14
[<] 14 15
[V] 15 16
[>] 16 17
[manager] 18 25
[\n] 25 26
L12
[\n] 1 2
L13
[BTree] 2 7
[(] 7 8
[)] 8 9
[{] 10 11
[\n] 11 12
L14
[instance] 3 11
[=] 12 13
[this] 14 18
[\n] 18 19
L15
[getLeaf] 3 10
[(] 10 11
[this] 11 15
[)] 15 16
[\n] 16 17
L16
[manager] 3 10
[=] 11 12
[new] 13 16
[BlockManager] 17 29
[<] 29 30
[>] 30 31
[(] 31 32
[)] 32 33
[\n] 33 34
L17
[}] 2 3
[\n] 3 4
L18
[\n] 1 2
L19
[// CPD-OFF] 2 12
L57
[\n] 11 12
L58
[\n] 1 2
L59
[def] 2 5
[add] 6 9
[(] 9 10
[K] 10 11
[key] 12 15
[,] 15 16
[V] 17 18
[value] 19 24
[)] 24 25
[{] 26 27
[\n] 27 28
L60
[\n] 1 2
L61
[if] 3 5
[(] 6 7
[count] 7 12
[>] 13 14
[0] 15 16
[&&] 17 19
[search] 20 26
[(] 26 27
[key] 27 30
[)] 30 31
[)] 31 32
[{] 33 34
[\n] 34 35
L62
[throw] 4 9
[new] 10 13
[IllegalArgumentException] 14 38
[(] 38 39
["$] 39 41
[key] 41 44
[is already in the tree"] 45 68
[)] 68 69
[\n] 69 70
L63
[}] 3 4
[\n] 4 5
L64
[\n] 1 2
L65
[BlockManager] 3 15
[.] 15 16
[Block] 16 21
[.] 21 22
[BlockElement] 22 34
[<] 34 35
[V] 35 36
[>] 36 37
[element] 38 45
[=] 46 47
[manager] 48 55
[.] 55 56
[element] 56 63
[\n] 63 64
L66
[element] 3 10
[.] 10 11
[value] 11 16
[=] 17 18
[value] 19 24
[\n] 24 25
L67
[\n] 1 2
L68
[super] 3 8
[.] 8 9
[add] 9 12
[key] 13 16
[,] 16 17
[element] 18 25
[\n] 25 26
L69
[}] 2 3
[\n] 3 4
L70
[\n] 1 2
L71
[def] 2 5
[delete] 6 12
[(] 12 13
[K] 13 14
[key] 15 18
[)] 18 19
[{] 20 21
[\n] 21 22
L72
[\n] 1 2
L73
[if] 3 5
[(] 6 7
[count] 7 12
[>] 13 14
[0] 15 16
[&&] 17 19
[!] 20 21
[search] 21 27
[(] 27 28
[key] 28 31
[)] 31 32
[)] 32 33
[{] 34 35
[\n] 35 36
L74
[throw] 4 9
[new] 10 13
[IllegalArgumentException] 14 38
[(] 38 39
["$] 39 41
[key] 41 44
[is not in the tree"] 45 64
[)] 64 65
[\n] 65 66
L75
[}] 3 4
[\n] 4 5
L76
[\n] 1 2
L77
[super] 3 8
[.] 8 9
[delete] 9 15
[key] 16 19
[\n] 19 20
L78
[\n] 1 2
L79
[if] 3 5
[(] 6 7
[count] 7 12
[==] 13 15
[0] 16 17
[)] 17 18
[{] 19 20
[\n] 20 21
L80
[getLeaf] 4 11
[(] 11 12
[this] 12 16
[)] 16 17
[\n] 17 18
L81
[}] 3 4
[\n] 4 5
L82
[}] 2 3
[\n] 3 4
L83
[}] 1 2
[\n] 2 3
EOF

174
pmd-groovy/src/test/resources/net/sourceforge/pmd/lang/groovy/cpd/testdata/sample.txt vendored
View File

File diff suppressed because it is too large Load Diff

4
pom.xml
View File

@ -742,9 +742,9 @@
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<groupId>org.apache.groovy</groupId>
<artifactId>groovy</artifactId>
<version>2.4.21</version>
<version>4.0.15</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>