aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorThomas Preud'homme <thomas.preudhomme@linaro.org>2018-07-06 11:43:49 +0100
committerThomas Preudhomme <thopre01@E119706.cambridge.arm.com>2018-12-20 22:53:09 +0000
commite0636da2dfd6d5e83f117a97e6671d12b79ef5db (patch)
treef17a1dc45bd05bd0a2d883059728c186ca9cebad
parent865c650f23b8ba2fd51e56852ea7aaae674f12ee (diff)
FileCheck: Add support for numeric variables and expressionslinaro-local/thomas.preudhomme/llvm-432-rebase
Summary: Sometimes there's a need to verify output which contain several numeric values that are related by a numeric expression. For example, one might want to check that two numbers are consecutive but accept any value for the smallest of the two. This patch introduces support for numeric variables and expressions to express this sort of constraint. A user can then define variables to have a numeric value and write numeric expression using those variable. A matching format specifier allows to match numbers in a variety of format: signed and unsigned decimal numbers as well as hex number in lower or capital case. For example, matching 2 consecutive signed decimal integers separated by a space is done via: [[%d,#N:]] [[#N+1]] The syntax is modeled after that of use and definition of the already supported variables flavour (renamed pattern variable for clarity) and pseudo variable by adding a # just after the double opening square bracket [[. Full details of the syntax can be found in the updated docs/CommandGuide/FileCheck.rst. One notable pitfall is that CHECK pattern with numeric expression using variable defined on the same line might fail to match despite there be a line which satisfy the conditions expressed by those numeric expression. This is due to lack of needed support from the regular expression engine that FileCheck relies on and is documented in the updated docs/CommandGuide/FileCheck.rst as well as possible workaround. Reviewers: jhenderson, chandlerc, jdenny, probinson, grimar, arichardson, rnk Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D49084
-rw-r--r--docs/CommandGuide/FileCheck.rst82
-rw-r--r--include/llvm/Support/FileCheck.h581
-rw-r--r--lib/Support/FileCheck.cpp1535
-rw-r--r--test/FileCheck/defines.txt48
-rw-r--r--test/FileCheck/numeric-expression.txt346
-rw-r--r--test/FileCheck/regex-scope.txt23
-rw-r--r--test/FileCheck/var-scope.txt26
-rw-r--r--test/FileCheck/verbose.txt33
8 files changed, 2398 insertions, 276 deletions
diff --git a/docs/CommandGuide/FileCheck.rst b/docs/CommandGuide/FileCheck.rst
index 721d2c2e782..a98d1ba99dd 100644
--- a/docs/CommandGuide/FileCheck.rst
+++ b/docs/CommandGuide/FileCheck.rst
@@ -102,8 +102,15 @@ and from the command line.
.. option:: -D<VAR=VALUE>
- Sets a filecheck variable ``VAR`` with value ``VALUE`` that can be used in
- ``CHECK:`` lines.
+ Sets a filecheck pattern variable ``VAR`` with value ``VALUE`` that can be
+ used in ``CHECK:`` lines.
+
+.. option:: -D#<FMT>,<NUMVAR>=<VALUE EXPRESSION>
+
+ Sets a filecheck numeric variable ``VAR`` of matching format ``FMT`` with the
+ value of evaluating ``VALUE EXPRESSION`` that can be used in ``CHECK:``
+ lines. See section ``FileCheck Numeric Variables and Expressions`` for
+ details on the format and meaning of <FMT> and <VALUE EXPRESSION>.
.. option:: -version
@@ -517,7 +524,7 @@ braces like you would in C. In the rare case that you want to match double
braces explicitly from the input, you can use something ugly like
``{{[{][{]}}`` as your pattern.
-FileCheck Variables
+FileCheck Pattern Variables
~~~~~~~~~~~~~~~~~~~
It is often useful to match a pattern and then verify that it occurs again
@@ -557,31 +564,86 @@ CHECK-LABEL block. Global variables are not affected by CHECK-LABEL.
This makes it easier to ensure that individual tests are not affected
by variables set in preceding tests.
-FileCheck Expressions
+FileCheck Numeric Variables and Expressions
~~~~~~~~~~~~~~~~~~~~~
-Sometimes there's a need to verify output which refers line numbers of the
+:program:`FileCheck` also allows to search for values satisfying a numeric
+expression constraint and define numeric variables accordingly. The numeric
+variables can themselves be used inside following numeric expressions. This
+allows to capture numeric relations between two numbers in a text to check,
+such as the need for consecutive registers to be used.
+
+The syntax for numeric expression is
+``[[#%<fmtspec>,<NUMVAR>: <constraint> <expr>]]`` where:
+* ``%<fmtspec>`` is an optional printf-style matching format specifier to
+ indicate what string representation the value to match should have (eg. hex
+ number) and conversely how to interpret a matched value into a value when
+ defining a numeric variable without a constraint expression (see below).
+ Currently accepted format specifier are ``%u``, ``%d``, ``%x`` and ``%X``.
+ If absent, the format specifier is inferred from the matching format of the
+ numeric variable(s) used by the expression constraint if any and they are all
+ the same, and default to ``%u`` if no numeric variable is used. In case of
+ conflict between matching format of several numeric variables the format
+ specifier is mandatory.
+* ``NUMVAR`` is an optional numeric variable to define to the value satisfying
+ the constraint.
+* ``<constraint>`` is the matching constraint describing how the value to match
+ must relate to the value of the numeric expression. The only currently
+ accepted constraint is ``==`` for an exact match and is the default if
+ ``<constraint>`` is not provided.
+* ``<expr>>`` is a numeric expression whose operands are either numeric
+ variables previously defined using the above syntax or a integer immediates.
+ Currently supported numeric operations are ``+`` and ``-``.
+
+For example:
+
+.. code-block:: llvm
+
+ ; CHECK: add r[[#REG:]], [[#REG]], [[#REG+1]]
+ ; CHECK: copying from [[#%x,ADDR:] to [[#ADDR + 8]]
+
+In the same way as for pattern variables, ``--enable-var-scope`` only consider
+global numeric variables that start with ``$`` and undefined local variables at
+the beginning of each CHECK-LABEL block.
+
+Important note: In its current implementation, a numeric expression using
+numeric variable defined on the same line can fail to match even though an
+input line satisfies it. This happens when there is more than one possible
+match when ignoring the constraint and considering only the matching format
+(ie. matching an unsigned number with [[:digit:]]+ and the first match does not
+satisfy the constraint. This is due to those numeric expressions constraints
+not being able to be expressed at the regular expression level and the regular
+expression engine not offering the possibility of getting all possible matches.
+
+FileCheck Pseudo Numeric Variables
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Sometimes there's a need to verify output which contain line numbers of the
match file, e.g. when testing compiler diagnostics. This introduces a certain
fragility of the match file structure, as "``CHECK:``" lines contain absolute
line numbers in the same file, which have to be updated whenever line numbers
change due to text addition or deletion.
-To support this case, FileCheck allows using ``[[@LINE]]``,
-``[[@LINE+<offset>]]``, ``[[@LINE-<offset>]]`` expressions in patterns. These
-expressions expand to a number of the line where a pattern is located (with an
-optional integer offset).
+To support this case, FileCheck understands the ``@LINE`` pseudo numeric
+variable which evaluate to the line number of the CHECK pattern where it is
+found.
This way match patterns can be put near the relevant test lines and include
relative line number references, for example:
.. code-block:: c++
- // CHECK: test.cpp:[[@LINE+4]]:6: error: expected ';' after top level declarator
+ // CHECK: test.cpp:[[# @LINE + 4]]:6: error: expected ';' after top level declarator
// CHECK-NEXT: {{^int a}}
// CHECK-NEXT: {{^ \^}}
// CHECK-NEXT: {{^ ;}}
int a
+To support legacy uses of ``@LINE`` as a special pattern variable,
+:program:`FileCheck` also accepts the following uses of ``@LINE`` with pattern
+variable syntax: ``[[@LINE]]``, ``[[@LINE+<offset>]]`` and
+``[[@LINE-<offset>]]`` where ``offset`` is an integer immediate.
+
Matching Newline Characters
~~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/include/llvm/Support/FileCheck.h b/include/llvm/Support/FileCheck.h
index 4061a26e22c..1ebeaee33b2 100644
--- a/include/llvm/Support/FileCheck.h
+++ b/include/llvm/Support/FileCheck.h
@@ -37,6 +37,428 @@ struct FileCheckRequest {
bool VerboseVerbose = false;
};
+//===----------------------------------------------------------------------===//
+// Numeric expression handling Code.
+//===----------------------------------------------------------------------===//
+
+struct FileCheckNumExprFmtType;
+
+/// Bitfield representing the format a numeric expression value should be
+/// printed into for matching. Used to represent both explicit format
+/// specifiers as well as implicit format from using numeric variables.
+struct FileCheckNumExprFmtType {
+ /// Value is signed.
+ unsigned Signed : 1;
+ /// Value should be print as hex number.
+ unsigned Hex : 1;
+ /// Capital case, only used for hex numbers.
+ unsigned Cap : 1;
+
+ /// When 0, absence of format, eg. for literals in an expression.
+ unsigned Set : 1;
+
+ /// Several conflicting implicit formats in an expression.
+ unsigned Conflict : 1;
+
+ /// Define format equality: formats are equal if all bits are identical.
+ bool operator==(const struct FileCheckNumExprFmtType &other);
+ bool operator!=(const struct FileCheckNumExprFmtType &other) {
+ return !(*this == other);
+ }
+};
+
+/// Initializer for numeric expression without format.
+const struct FileCheckNumExprFmtType FmtNone = {0, 0, 0, 0, 0};
+/// Initializer for numeric expression matched as unsigned value.
+const struct FileCheckNumExprFmtType FmtUnsigned = {0, 0, 0, 1, 0};
+/// Initializer for numeric expression matched as signed value.
+const struct FileCheckNumExprFmtType FmtSigned = {1, 0, 0, 1, 0};
+/// Initializer for numeric expression matched as lower case hex value.
+const struct FileCheckNumExprFmtType FmtLowHex = {0, 1, 0, 1, 0};
+/// Initializer for numeric expression matched as capital case hex value.
+const struct FileCheckNumExprFmtType FmtCapHex = {0, 1, 1, 1, 0};
+
+namespace Check {
+/// Phases of FileCheck. Used to determine at what phase is a numeric
+/// expression (and thus value of any potential variable defined by it)
+/// available.
+enum FileCheckPhase {
+ /// Parsing the CHECK pattern.
+ ParsePhase = 0,
+ /// Pattern substitution and actual regex matching
+ MatchPhase,
+ /// Constraint check done for numeric expression just after match phase.
+ CheckPhase
+};
+} // namespace Check
+
+/// Class representing a numeric expression's value. Used to be able to store
+/// and return both signed and unsigned value.
+class FileCheckNumExprVal {
+private:
+ /// Signed value.
+ int64_t SVal;
+
+ /// Unsigned value.
+ uint64_t UVal;
+
+ /// Whether value is signed (and thus is stored in SVal) or not (in which
+ /// case it is stored in UVal).
+ bool Signed;
+
+ /// Whether this holds an actual value. Examples of where this would be
+ /// false include:
+ /// - underflow or overflow of one of the binary operation in the expression
+ /// - value of undefined variable
+ bool Valid;
+
+ /// Whether this is a tentative value. True if any of the operand involved
+ /// in its computation had a tentative value.
+ bool Tentative;
+
+public:
+ /// Constructor for an invalid value.
+ FileCheckNumExprVal() : Valid(false) {}
+
+ /// Constructor for a signed value.
+ FileCheckNumExprVal(int64_t Val)
+ : SVal(Val), Signed(true), Valid(true), Tentative(false) {}
+
+ /// Constructor for an unsigned value.
+ FileCheckNumExprVal(uint64_t Val)
+ : UVal(Val), Signed(false), Valid(true), Tentative(false) {}
+
+ /// Define how to compare value. Used to decide whether a matched numeric
+ /// value satisfies its corresponding numeric expression.
+ bool operator==(const FileCheckNumExprVal &other);
+ bool operator!=(const FileCheckNumExprVal &other) {
+ return !(*this == other);
+ }
+
+ /// Whether value is signed.
+ bool IsSigned() const { return Signed; }
+
+ /// Whether value is valid.
+ bool IsValid() const { return Valid; }
+
+ /// Whether value is tentative.
+ bool IsTentative() const { return Tentative; }
+
+ /// Return the signed value. Must only be called if value is signed in the
+ /// first place.
+ int64_t GetSignedValue() const { return SVal; }
+
+ /// Return the unsigned value. Must only be called if value is unsigned in
+ /// the first place.
+ uint64_t GetUnsignedValue() const { return UVal; }
+
+ /// Mark value as tentative.
+ bool ToggleTentative() { return Tentative = !Tentative; }
+
+ /// Convert value to a signed value, or mark value invalid if not possible
+ /// (original value was not within range for a signed integer).
+ void ConvertSigned();
+
+ /// Convert value to an unsigned value, or mark value invalid if not possible
+ /// (original value was not within range for an unsigned integer).
+ void ConvertUnsigned();
+
+ /// Store in \p StringRepr a string representation of this value given the
+ /// matching format \p Fmt. Return whether value had no such a string
+ /// representation (true if value is invalid).
+ bool GetStringRepr(struct FileCheckNumExprFmtType Fmt,
+ std::string &StringRepr) const;
+
+ /// Perform an addition operation. Return an invalid value in case of
+ /// underflow or overflow.
+ static FileCheckNumExprVal Add(const FileCheckNumExprVal &Op1,
+ const FileCheckNumExprVal &Op2);
+
+ /// Perform a substraction operation. Return an invalid value in case of
+ /// underflow or overflow.
+ static FileCheckNumExprVal Sub(const FileCheckNumExprVal &Op1,
+ const FileCheckNumExprVal &Op2);
+};
+
+/// Base class representing the AST of a given numeric expression. Only used
+/// for AST node of unknown type. All actual nodes are of another type.
+class FileCheckNumExprAST {
+public:
+ virtual ~FileCheckNumExprAST() = default;
+
+ /// Evaluate the value of the expression represented by this AST. Return in
+ /// \p Fmt the implicit conversion that applies from any variable used or
+ /// null if no variable is used by the expression. This method must be
+ /// overrided in all subclasses.
+ virtual FileCheckNumExprVal Eval(struct FileCheckNumExprFmtType &Fmt) = 0;
+
+ /// Append names of undefined variables used in the expression represented by
+ /// this AST. Must be overrided in any subclass representing an expression
+ /// that can contain a variable.
+ virtual void GetUndefVarNames(std::vector<StringRef> &UndefVarNames) const {}
+};
+
+/// Class representing a litteral, either in the AST of a numeric expression or
+/// in the result of the evaluation of such an expression.
+class FileCheckNumExprLiteral : public FileCheckNumExprAST {
+private:
+ /// Actual value of the literal.
+ FileCheckNumExprVal Value;
+
+public:
+ /// Constructor for a signed literal.
+ FileCheckNumExprLiteral(int64_t Val) : Value(Val) {}
+
+ /// Constructor for an unsigned literal.
+ FileCheckNumExprLiteral(uint64_t Val) : Value(Val) {}
+
+ /// Evaluate the value of this literal. Therefore returns this node itself
+ /// and set \p Fmt to null since literals do not carry any implicit
+ /// conversion.
+ FileCheckNumExprVal Eval(struct FileCheckNumExprFmtType &Fmt);
+};
+
+/// Class representing a numeric expression and its matching format.
+class FileCheckNumExpr {
+private:
+ /// Pointer to AST of the numeric expression.
+ std::shared_ptr<FileCheckNumExprAST> AST;
+
+ /// Value of the numeric expression. Set either from evaluating the AST if
+ /// any and using only variable defined on previous CHECK line, or from the
+ /// matched value if constraint is satisfied.
+ FileCheckNumExprVal Value;
+
+ /// Matching format.
+ struct FileCheckNumExprFmtType Fmt;
+
+ /// FileCheck phase when this numeric expression can be evaluated.
+ enum Check::FileCheckPhase KnownPhase;
+
+public:
+ /// Generic constructor for a numeric expression whose equality constraint is
+ /// represented by \p AST, matching format is \p Fmt and whose AST can be
+ /// evaluated in phase \p KnownPhase.
+ FileCheckNumExpr(std::shared_ptr<FileCheckNumExprAST> AST,
+ struct FileCheckNumExprFmtType Fmt,
+ enum Check::FileCheckPhase KnownPhase);
+
+ /// Constructor for numeric expression with a known value in parse phase,
+ /// eg. the numeric expression defining the @LINE numeric variable (and
+ /// currently only used for that).
+ FileCheckNumExpr(FileCheckNumExprVal Value,
+ struct FileCheckNumExprFmtType Fmt);
+
+ /// Return pointer to AST of the numeric expression. Pointed-to AST is
+ /// managed by a shared_ptr and is guaranteed live as long as this object is.
+ FileCheckNumExprAST *GetAST() const { return AST.get(); }
+
+ /// Return the matched value for the expression.
+ FileCheckNumExprVal GetValue() const { return Value; }
+
+ /// Set Value to \p Val if currently invalid or tentative. Return whether
+ /// Value failed to be set (Value was already valid and final).
+ bool SetTentativeValue(FileCheckNumExprVal Val);
+
+ /// Return matching format of the expression.
+ struct FileCheckNumExprFmtType GetFormat() const {
+ return Fmt;
+ }
+
+ /// Return at what FileCheck phase can this expression can be evaluated.
+ enum Check::FileCheckPhase GetKnownPhase() { return KnownPhase; }
+
+ /// Store in \p Val the value corresponding to string representation
+ /// \p StrVal according to matching format of this numeric expression.
+ /// Return whether \p StrVal correspond to a valid and representable value.
+ bool ValueFromStringRepr(StringRef StrVal, FileCheckNumExprVal &Val) const;
+
+ /// Store in \p ValueStringRepr the string representation of Value given the
+ /// matching format of this numeric expression. Return whether Value does
+ /// not have a string representation (Value is invalid).
+ bool GetValueStringRepr(std::string &ValueStringRepr) const {
+ return Value.GetStringRepr(Fmt, ValueStringRepr);
+ }
+
+ /// Store in \p MatchString the regexp pattern to use to match this
+ /// numeric expression given its matching format. Return whether the
+ /// function was unable to determine the regexp pattern.
+ bool GetMatchString(enum Check::FileCheckPhase Phase,
+ std::string &MatchString);
+
+ /// Verify that the matched value in \p MatchedValue satisfies the constraint
+ /// expressed by this expression. Return true if constraint is not
+ /// satisfied.
+ bool VerifyConstraint(FileCheckNumExprVal MatchedValue) const;
+
+ /// Store in \p EvaluatedValue the value evaluated from the constraint of
+ /// this numeric expression. Return whether evaluation failed.
+ bool Eval(FileCheckNumExprVal &EvaluatedValue) const;
+
+ /// Set Value from the result of calling Eval(). Return whether it failed.
+ bool SetValueFromEval();
+
+ /// Mark value as final. Return whether value was already final.
+ bool CommitValue();
+};
+
+/// Class representing a numeric variable with a given value in the AST of a
+/// numeric expression. Each definition of a variable gets its own instance
+/// of this class, uses share the same instance as the respective definition.
+class FileCheckNumExprVar : public FileCheckNumExprAST {
+private:
+ /// Name of the numeric variable.
+ StringRef Name;
+
+ /// Pointer to numeric expression defining this numeric variable. Only null
+ /// if variable used but not defined. If numeric expression is empty NumExpr
+ /// points to a FileCheckNumExpr with a null AST.
+ std::shared_ptr<FileCheckNumExpr> NumExpr;
+
+ /// Line number where this variable is defined. Used to determine whether a
+ /// variable is defined on the same line as a given use.
+ unsigned DefLineNumber;
+
+public:
+ /// Constructor for an undefined variable \p Name.
+ FileCheckNumExprVar(StringRef Name)
+ : Name(Name), NumExpr(nullptr), DefLineNumber(0) {}
+
+ /// Constructor for a variable \p Name defined at line \p DefLineNumber.
+ FileCheckNumExprVar(StringRef Name, unsigned DefLineNumber)
+ : Name(Name), NumExpr(nullptr), DefLineNumber(DefLineNumber) {}
+
+ /// Constructor for numeric variable \p Name with a known \p Value at parse
+ /// time (eg. the @LINE numeric variable). Matching format of the variable
+ /// is given in \p Fmt and \p DefLineNumber indicates when does this variable
+ /// starts to be defined.
+ FileCheckNumExprVar(StringRef Name, FileCheckNumExprVal Value,
+ struct FileCheckNumExprFmtType Fmt,
+ unsigned DefLineNumber);
+
+ /// Return name of that numeric variable.
+ StringRef GetName() const { return Name; }
+
+ /// Return pointer to the numeric expression defining this numeric variable.
+ FileCheckNumExpr *GetNumExpr() const { return NumExpr.get(); }
+
+ /// Return line number where this variable is defined.
+ unsigned GetDefLineNumber() { return DefLineNumber; }
+
+ /// Set pointer to numeric expression defining this numeric variable.
+ void SetNumExpr(std::shared_ptr<FileCheckNumExpr> NumExpr) {
+ this->NumExpr = NumExpr;
+ }
+
+ /// Evaluate the value of this numeric variable. Therefore returns the value
+ /// and implicit conversion of this numeric variable stored in its
+ /// corresponding numeric expression class.
+ FileCheckNumExprVal Eval(struct FileCheckNumExprFmtType &Fmt);
+
+ /// Append numeric variable's name to UndefVarNames if undefined.
+ void GetUndefVarNames(std::vector<StringRef> &UndefVarNames) const;
+};
+
+/// Type of functions evaluating a given binary operation.
+using binop_eval_t = FileCheckNumExprVal (*)(const FileCheckNumExprVal &,
+ const FileCheckNumExprVal &);
+
+/// Class representing a single binary operation in the AST of a numeric
+/// expression.
+class FileCheckASTBinop : public FileCheckNumExprAST {
+private:
+ /// Left operand.
+ std::shared_ptr<FileCheckNumExprAST> Opl;
+
+ /// Right operand.
+ std::shared_ptr<FileCheckNumExprAST> Opr;
+
+ /// Pointer to function that can evaluate this binary operation.
+ binop_eval_t EvalBinop;
+
+public:
+ FileCheckASTBinop(binop_eval_t EvalBinop,
+ std::shared_ptr<FileCheckNumExprAST> OperandLeft,
+ std::shared_ptr<FileCheckNumExprAST> OperandRight)
+ : Opl(OperandLeft), Opr(OperandRight), EvalBinop(EvalBinop) {}
+
+ /// Evaluate the value of the binary operation represented by this AST. Uses
+ /// EvalBinop to perform the binary operation on the values of recursively
+ /// evaluating the left and right operands.
+ FileCheckNumExprVal Eval(struct FileCheckNumExprFmtType &Fmt);
+
+ /// Append to UnderVarNames the names of undefined numeric variables used in
+ /// any of the operands.
+ void GetUndefVarNames(std::vector<StringRef> &UndefVarNames) const;
+};
+
+class FileCheckPatternContext;
+
+/// Class representing a substitution to perform in the string to match, either
+/// a pattern variable to replace by its value or a numeric expression to
+/// replace by an appropriate wildcard pattern.
+class FileCheckPatternSubst {
+private:
+ /// Pointer to a class instance holding among other thing the table with the
+ /// values of live pattern variables at the start of any given CHECK line.
+ /// Used for substituting pattern variables (numeric variables have their
+ /// value in the FileCheckNumExpr class instance pointed to by NumExpr).
+ FileCheckPatternContext *Context;
+
+ /// Whether this represents a numeric expression substitution.
+ bool isNumExpr;
+
+ /// The string that needs to be substituted for something else. For a
+ /// pattern variable this is its name, otherwise this is the whole numeric
+ /// expression.
+ StringRef SubstStr;
+
+ /// If this is a numeric expression substitution, this is the pointer to the
+ /// class representing that numeric expression.
+ std::shared_ptr<FileCheckNumExpr> NumExpr;
+
+ // Index in RegExStr of where to do the substitution.
+ unsigned InsertIdx;
+
+public:
+ /// Constructor for a pattern variable substitution.
+ FileCheckPatternSubst(FileCheckPatternContext *Context, StringRef VarName,
+ unsigned InsertIdx)
+ : Context(Context), isNumExpr(false), SubstStr(VarName),
+ InsertIdx(InsertIdx) {}
+
+ /// Constructor for a numeric expression substitution.
+ FileCheckPatternSubst(FileCheckPatternContext *Context, StringRef Expr,
+ std::shared_ptr<FileCheckNumExpr> NumExpr,
+ unsigned InsertIdx)
+ : Context(Context), isNumExpr(true), SubstStr(Expr), NumExpr(NumExpr),
+ InsertIdx(InsertIdx) {}
+
+ /// Return whether this is a numeric expression substitution.
+ bool IsNumExpr() const { return isNumExpr; }
+
+ /// Return the string to be substituted.
+ StringRef GetSubstString() const { return SubstStr; }
+
+ /// Return the index where the substitution is to be performed.
+ unsigned GetIndex() const { return InsertIdx; }
+
+ /// Perform in \p SubstValue the substitution represented by this class
+ /// instance. For a numeric variable we replace it by its value if known at
+ /// match time or a suitable wildcard pattern otherwise. For a pattern
+ /// variable we simply replace it by the text its definition matched.
+ /// Return whether substitution failed.
+ bool Substitute(std::string &SubstValue) const;
+
+ /// Write in \p MatchedValue the value successfully matched (ie. constraint
+ /// is verified for a numeric expression) by the substitution. Return
+ /// whether match failed for that substitution.
+ bool MatchedValue(std::string &MatchedValue) const;
+
+ /// Append to UnderVarNames the names of undefined pattern or numeric
+ /// variables used in this substitution.
+ void GetUndefVarNames(std::vector<StringRef> &UndefVarNames) const;
+};
//===----------------------------------------------------------------------===//
// Pattern Handling Code.
@@ -80,7 +502,60 @@ public:
std::string getDescription(StringRef Prefix) const;
};
-}
+} // namespace Check
+
+/// Structure representing the definition of a numeric variable in a pattern.
+/// It holds the parenthesized capture number and the pointer to the class
+/// representing the numeric expression and its matching format.
+struct FileCheckNumExprMatch {
+ /// Pointer to class representing the numeric expression and its matching
+ /// format.
+ std::shared_ptr<FileCheckNumExpr> NumExpr;
+
+ /// Parenthesized capture number for this numeric expression
+ unsigned CaptureParen;
+};
+
+/// Class holding the FileCheckPattern global state, shared by all patterns:
+/// tables holding values of variables and whether they are defined or not at
+/// any given time in the matching process.
+class FileCheckPatternContext {
+ friend class FileCheckPattern;
+
+private:
+ /// When matching a given pattern, this holds the value of all the FileCheck
+ /// pattern variables defined in previous patterns. In a pattern only the
+ /// last definition for a given variable is recorded in this table,
+ /// back-references are used for uses after any the other definition.
+ StringMap<StringRef> GlobalVariableTable;
+
+ /// Map of all pattern variables defined so far. Used at parse time to
+ /// detect a name conflict between a numeric variable and a pattern variable
+ /// when the former is defined on a later line than the latter.
+ StringMap<bool> DefinedVariableTable;
+
+ /// When matching a given pattern, this holds the pointers to the classes
+ /// representing the AST of all numeric variables defined in previous
+ /// patterns along with their values. In a pattern only the last definition
+ /// for a given variable is recorded in this table. When matching a pattern
+ /// all definitions for that pattern are recorded in NumericVariableDefs
+ /// table.
+ StringMap<std::shared_ptr<FileCheckNumExprVar>> GlobalNumericVariableTable;
+
+public:
+ /// Store in \p VarValue the value of pattern variable \p VarName.
+ bool GetPatternVarValue(StringRef VarName, StringRef &VarValue);
+
+ /// Define pattern and numeric variables from definitions given on the
+ /// command line passed as a vector of VAR=VAL strings in \p CmdlineDefines.
+ /// Report any error to \p SM.
+ bool DefineCmdlineVariables(std::vector<std::string> &CmdlineDefines,
+ SourceMgr &SM);
+
+ /// Undefine local variables (variables whose name does not start with a '$'
+ /// sign), ie remove them from GlobalVariableTable.
+ void ClearLocalVars(void);
+};
struct FileCheckDiag;
@@ -95,43 +570,81 @@ class FileCheckPattern {
/// a fixed string to match.
std::string RegExStr;
- /// Entries in this vector map to uses of a variable in the pattern, e.g.
- /// "foo[[bar]]baz". In this case, the RegExStr will contain "foobaz" and
- /// we'll get an entry in this vector that tells us to insert the value of
- /// bar at offset 3.
- std::vector<std::pair<StringRef, unsigned>> VariableUses;
-
- /// Maps definitions of variables to their parenthesized capture numbers.
+ /// Entries in this vector represent uses of a pattern variable or a numeric
+ /// expression in the pattern that need to be substituted in the regexp
+ /// pattern at match time, e.g. "foo[[bar]]baz[[#N+1]]". In this case, the
+ /// RegExStr will contain "foobaz" and we'll get two entries in this vector
+ /// that tells us to insert the value of pattern variable "bar" at offset 3
+ /// and the value of numeric expression "N+1" at offset 6. Uses are
+ /// represented by a FileCheckPatternSubst class to abstract whether it is a
+ /// pattern variable or a numeric expression.
+ std::vector<FileCheckPatternSubst> MatchSubsts;
+
+ /// Entries in this vector represent any use of a pattern variable or a
+ /// numeric expression in the pattern. Substitution are represented in the
+ /// same way as in MatchSubsts above.
+ std::vector<FileCheckPatternSubst> AllSubsts;
+
+ /// Maps names of pattern variables defined in a pattern to the parenthesized
+ /// capture numbers of their last definition.
+ ///
+ /// E.g. for the pattern "foo[[bar:.*]]baz[[bar]]quux[[bar:.*]]",
+ /// VariableDefs will map "bar" to 2 corresponding to the second definition
+ /// of "bar".
///
- /// E.g. for the pattern "foo[[bar:.*]]baz", VariableDefs will map "bar" to
- /// 1.
+ /// Note: uses std::map rather than StringMap to be able to get the key when
+ /// iterating over values.
std::map<StringRef, unsigned> VariableDefs;
+ /// Holds the capture parentheses number and pointer to corresponding
+ /// FileCheckNumExpr class instance of all numeric expressions whose matching
+ /// result must be captured, that is numeric expressions with a variable
+ /// definition or using numeric variable defined on the same line. Used to
+ /// record the matched value of all those expressions and for those with a
+ /// constraint to check that the matched value satisfies it.
+ std::vector<FileCheckNumExprMatch> CapturedNumericExpressions;
+
+ /// Pointer to a class instance holding the global state shared by all
+ /// patterns:
+ /// - tables with the values of live pattern and numeric variables at
+ /// the start of any given CHECK line;
+ /// - table holding whether a pattern variable has been defined at any given
+ /// point during the parsing phase.
+ FileCheckPatternContext *Context;
+
Check::FileCheckType CheckTy;
- /// Contains the number of line this pattern is in.
+ /// Line number for this CHECK pattern. Used to determine whether a variable
+ /// definition is made on an earlier line to the one with this CHECK.
unsigned LineNumber;
public:
- explicit FileCheckPattern(Check::FileCheckType Ty)
- : CheckTy(Ty) {}
+ explicit FileCheckPattern(Check::FileCheckType Ty,
+ FileCheckPatternContext *Context)
+ : Context(Context), CheckTy(Ty) {}
/// Returns the location in source code.
SMLoc getLoc() const { return PatternLoc; }
+ /// Returns the pointer to the global state for all patterns in this
+ /// FileCheck instance.
+ FileCheckPatternContext *GetContext() const { return Context; }
+
+ bool ParseVariable(StringRef Str, bool &IsPseudo, unsigned &TrailIdx) const;
+ std::shared_ptr<FileCheckNumExpr>
+ ParseNumericExpression(StringRef Expr,
+ std::shared_ptr<FileCheckNumExprVar> &NumVarDef,
+ const SourceMgr &SM) const;
bool ParsePattern(StringRef PatternStr, StringRef Prefix, SourceMgr &SM,
unsigned LineNumber, const FileCheckRequest &Req);
- size_t Match(StringRef Buffer, size_t &MatchLen,
- StringMap<StringRef> &VariableTable) const;
- void PrintVariableUses(const SourceMgr &SM, StringRef Buffer,
- const StringMap<StringRef> &VariableTable,
- SMRange MatchRange = None) const;
+ size_t Match(StringRef Buffer, size_t &MatchLen, const SourceMgr &SM) const;
+ void PrintSubsts(const SourceMgr &SM, StringRef Buffer,
+ SMRange MatchRange = None) const;
void PrintFuzzyMatch(const SourceMgr &SM, StringRef Buffer,
- const StringMap<StringRef> &VariableTable,
std::vector<FileCheckDiag> *Diags) const;
bool hasVariable() const {
- return !(VariableUses.empty() && VariableDefs.empty());
+ return !(AllSubsts.empty() && VariableDefs.empty());
}
Check::FileCheckType getCheckTy() const { return CheckTy; }
@@ -141,11 +654,25 @@ public:
private:
bool AddRegExToRegEx(StringRef RS, unsigned &CurParen, SourceMgr &SM);
void AddBackrefToRegEx(unsigned BackrefNum);
- unsigned
- ComputeMatchDistance(StringRef Buffer,
- const StringMap<StringRef> &VariableTable) const;
- bool EvaluateExpression(StringRef Expr, std::string &Value) const;
+ unsigned ComputeMatchDistance(StringRef Buffer) const;
size_t FindRegexVarEnd(StringRef Str, SourceMgr &SM);
+
+ /// Numeric expression parsing helpers.
+ std::shared_ptr<FileCheckNumExprVar>
+ ParseNumericVariable(StringRef &Expr, bool IsDefinition,
+ const SourceMgr &SM) const;
+ std::shared_ptr<FileCheckNumExprAST>
+ ParseNumericOperand(StringRef &Expr, const SourceMgr &SM,
+ struct FileCheckNumExprFmtType &ImplicitFmt,
+ enum Check::FileCheckPhase &KnownPhase) const;
+ std::shared_ptr<FileCheckNumExprAST>
+ ParseFileCheckBinop(StringRef &Expr, std::shared_ptr<FileCheckNumExprAST> Opl,
+ const SourceMgr &SM,
+ struct FileCheckNumExprFmtType &ImplicitFmt,
+ enum Check::FileCheckPhase &KnownPhase) const;
+ std::shared_ptr<FileCheckNumExpr>
+ ParseLegacyNumericExpression(StringRef Expr, StringRef Name,
+ StringRef Trailer, const SourceMgr &SM) const;
};
//===----------------------------------------------------------------------===//
@@ -224,19 +751,17 @@ struct FileCheckString {
: Pat(P), Prefix(S), Loc(L) {}
size_t Check(const SourceMgr &SM, StringRef Buffer, bool IsLabelScanMode,
- size_t &MatchLen, StringMap<StringRef> &VariableTable,
- FileCheckRequest &Req, std::vector<FileCheckDiag> *Diags) const;
+ size_t &MatchLen, FileCheckRequest &Req,
+ std::vector<FileCheckDiag> *Diags) const;
bool CheckNext(const SourceMgr &SM, StringRef Buffer) const;
bool CheckSame(const SourceMgr &SM, StringRef Buffer) const;
bool CheckNot(const SourceMgr &SM, StringRef Buffer,
const std::vector<const FileCheckPattern *> &NotStrings,
- StringMap<StringRef> &VariableTable,
const FileCheckRequest &Req,
std::vector<FileCheckDiag> *Diags) const;
size_t CheckDag(const SourceMgr &SM, StringRef Buffer,
std::vector<const FileCheckPattern *> &NotStrings,
- StringMap<StringRef> &VariableTable,
const FileCheckRequest &Req,
std::vector<FileCheckDiag> *Diags) const;
};
diff --git a/lib/Support/FileCheck.cpp b/lib/Support/FileCheck.cpp
index 37986c96c08..f0e2f32e55f 100644
--- a/lib/Support/FileCheck.cpp
+++ b/lib/Support/FileCheck.cpp
@@ -25,6 +25,829 @@
using namespace llvm;
+/// Define format equality: formats are equal if all bits are identical.
+bool FileCheckNumExprFmtType::
+operator==(const struct FileCheckNumExprFmtType &other) {
+ return Set == other.Set && Conflict == other.Conflict &&
+ Signed == other.Signed && Hex == other.Hex && Cap == other.Cap;
+}
+
+/// Equality operator: 2 values are equal if both are valid and have same
+/// signedness and corresponding value.
+bool FileCheckNumExprVal::operator==(const FileCheckNumExprVal &other) {
+ if (!Valid || !other.Valid)
+ return false;
+
+ if (Signed)
+ return SVal == other.SVal;
+ else
+ return UVal == other.UVal;
+}
+
+/// Convert value to a signed value, or mark value invalid if not possible
+/// (original value was not within range for a signed integer).
+void FileCheckNumExprVal::ConvertSigned() {
+ if (!Valid || Signed)
+ return;
+
+ if (UVal > std::numeric_limits<int64_t>::max()) {
+ Valid = false;
+ return;
+ }
+
+ SVal = UVal;
+ Signed = true;
+}
+
+/// Convert value to an unsigned value, or mark value invalid if not possible
+/// (original value was not within range for an unsigned integer).
+void FileCheckNumExprVal::ConvertUnsigned() {
+ if (!Valid || !Signed)
+ return;
+
+ if (SVal < 0) {
+ Valid = false;
+ return;
+ }
+
+ UVal = SVal;
+ Signed = false;
+ return;
+}
+
+/// Store in \p StringRepr a string representation of this value given the
+/// matching format \p Fmt. Return whether value had no such a string
+/// representation (true if value is invalid).
+bool FileCheckNumExprVal::GetStringRepr(struct FileCheckNumExprFmtType Fmt,
+ std::string &StringRepr) const {
+ if (!Valid)
+ return true;
+
+ if (Fmt.Hex) {
+ StringRepr = utohexstr(GetUnsignedValue(), !Fmt.Cap);
+ } else if (Fmt.Signed)
+ StringRepr = itostr(GetSignedValue());
+ else
+ StringRepr = utostr(GetUnsignedValue());
+
+ return false;
+}
+
+/// Perform an addition operation. Return an invalid value in case of
+/// underflow or overflow.
+FileCheckNumExprVal FileCheckNumExprVal::Add(const FileCheckNumExprVal &Op1,
+ const FileCheckNumExprVal &Op2) {
+ // Operands must be valid.
+ if (!Op1.Valid || !Op2.Valid)
+ return FileCheckNumExprVal();
+
+ // Operands must have same sign.
+ if (Op1.Signed != Op2.Signed)
+ return FileCheckNumExprVal();
+
+ if (Op1.Signed) {
+ int64_t Val1 = Op1.SVal;
+ int64_t Val2 = Op2.SVal;
+
+ // Op1 + Op2 > max int64_t.
+ if (Val1 > 0 && Val2 > 0 &&
+ Val1 > (std::numeric_limits<int64_t>::max() - Val2))
+ return FileCheckNumExprVal();
+
+ // Op1 + Op2 < min int64_t.
+ if (Val1 < 0 && Val2 < 0 &&
+ Val1 < (std::numeric_limits<int64_t>::min() - Val2))
+ return FileCheckNumExprVal();
+
+ return FileCheckNumExprVal(Val1 + Val2);
+ } else {
+ uint64_t Val1 = Op1.UVal;
+ uint64_t Val2 = Op2.UVal;
+
+ // Op1 + Op2 > max uint64_t.
+ if (Val1 > std::numeric_limits<uint64_t>::max() - Val2)
+ return FileCheckNumExprVal();
+ return FileCheckNumExprVal(Val1 + Val2);
+ }
+}
+
+/// Perform a substraction operation. Return an invalid value in case of
+/// underflow or overflow.
+FileCheckNumExprVal FileCheckNumExprVal::Sub(const FileCheckNumExprVal &Op1,
+ const FileCheckNumExprVal &Op2) {
+ // Operands must be valid.
+ if (!Op1.Valid || !Op2.Valid)
+ return FileCheckNumExprVal();
+
+ // Operands must have same sign.
+ if (Op1.Signed != Op2.Signed)
+ return FileCheckNumExprVal();
+
+ if (Op1.Signed) {
+ int64_t Val1 = Op1.SVal;
+ int64_t Val2 = Op2.SVal;
+
+ // Op1 - Op2 > max int64_t.
+ if (Val1 > 0 && Val2 < 0 &&
+ Val1 > (std::numeric_limits<int64_t>::max() + Val2))
+ return FileCheckNumExprVal();
+
+ // Op1 - Op2 < min int64_t.
+ if (Val1 < 0 && Val2 > 0 &&
+ Val1 < (std::numeric_limits<int64_t>::min() + Val2))
+ return FileCheckNumExprVal();
+
+ return FileCheckNumExprVal(Val1 - Val2);
+ } else {
+ uint64_t Val1 = Op1.UVal;
+ uint64_t Val2 = Op2.UVal;
+
+ // Op1 < Op2.
+ if (Val1 < Val2)
+ return FileCheckNumExprVal();
+ return FileCheckNumExprVal(Val1 - Val2);
+ }
+}
+
+/// Evaluate the value of this literal. Therefore returns this node itself and
+/// set \p Fmt to null since literals do not carry any implicit conversion.
+FileCheckNumExprVal
+FileCheckNumExprLiteral::Eval(struct FileCheckNumExprFmtType &Fmt) {
+ Fmt = FmtNone;
+ return Value;
+}
+
+/// Generic constructor for a numeric expression whose equality constraint is
+/// represented by \p AST, matching format is \p Fmt and whose AST can be
+/// evaluated in phase \p KnownPhase. If matching format is unset (ie. no
+/// explicit or implicit matching format), set it to default one (unsigned
+/// decimal integer).
+FileCheckNumExpr::FileCheckNumExpr(std::shared_ptr<FileCheckNumExprAST> AST,
+ struct FileCheckNumExprFmtType Fmt,
+ enum Check::FileCheckPhase KnownPhase)
+ : AST(AST), KnownPhase(KnownPhase) {
+ if (!Fmt.Set)
+ this->Fmt = FmtUnsigned;
+ else
+ this->Fmt = Fmt;
+}
+
+/// Constructor for numeric expression with a known value in parse phase,
+/// eg. the numeric expression defining the @LINE numeric variable (and
+/// currently only used for that).
+FileCheckNumExpr::FileCheckNumExpr(FileCheckNumExprVal Value,
+ struct FileCheckNumExprFmtType Fmt)
+ : FileCheckNumExpr(nullptr, Fmt, Check::ParsePhase) {
+ this->Value = Value;
+ this->KnownPhase = Check::ParsePhase;
+}
+
+/// Set Value to \p Val if currently invalid or tentative. Return whether
+/// Value failed to be set (Value was already valid and final).
+bool FileCheckNumExpr::SetTentativeValue(FileCheckNumExprVal Val) {
+ // Allow a value to be tentatively set several time.
+ if (Value.IsValid() && !Value.IsTentative())
+ return true;
+
+ // Value should be set from evaluating the AST if any.
+ if (AST != nullptr)
+ return true;
+
+ assert(!Val.IsTentative() && "Setting from already tentative matched value");
+ Val.ToggleTentative();
+ Value = Val;
+ return false;
+}
+
+/// Store in \p Val the value corresponding to string representation \p StrVal
+/// according to matching format of this numeric expression. Return whether
+/// \p StrVal correspond to a valid and representable value.
+bool FileCheckNumExpr::ValueFromStringRepr(StringRef StrVal,
+ FileCheckNumExprVal &Val) const {
+ unsigned Radix = Fmt.Hex ? 16 : 10;
+ if (Fmt.Signed) {
+ int64_t SVal;
+
+ if (StrVal.getAsInteger(Radix, SVal))
+ return true;
+
+ Val = FileCheckNumExprVal(SVal);
+ } else {
+ uint64_t UVal;
+
+ if (StrVal.getAsInteger(Radix, UVal))
+ return true;
+
+ Val = FileCheckNumExprVal(UVal);
+ }
+
+ return false;
+}
+
+/// Store in \p MatchString the regexp pattern to use to match this numeric
+/// expression given its matching format. Return whether the function was
+/// unable to determine the regexp pattern.
+bool FileCheckNumExpr::GetMatchString(enum Check::FileCheckPhase Phase,
+ std::string &MatchString) {
+ assert(Phase <= Check::MatchPhase && "Asking match string after match done");
+
+ // We know the value, return its string representation. If supporting other
+ // constraints than equality this should be restricted to equality only.
+ if (Phase >= KnownPhase) {
+ if (!Value.IsValid())
+ return true;
+
+ if (Value.GetStringRepr(Fmt, MatchString))
+ return true;
+ return false;
+ // We do not know the value but we will at match time.
+ } else if (KnownPhase <= Check::MatchPhase) {
+ MatchString = std::string();
+ return false;
+ // We do not know the value and will not know it at match time. Output
+ // appropriate wildcard pattern.
+ } else {
+ assert(Fmt.Set && !(Fmt.Hex && Fmt.Signed) &&
+ "Numeric expression with unexpected conversion");
+ if (Fmt.Hex) {
+ if (Fmt.Cap)
+ MatchString = std::string("[[:digit:]A-F]+");
+ else
+ MatchString = std::string("[[:digit:]a-f]+");
+ } else if (Fmt.Signed)
+ MatchString = std::string("-?[[:digit:]]+");
+ else
+ MatchString = std::string("[[:digit:]]+");
+
+ return false;
+ }
+}
+
+/// Verify that the matched value in \p MatchedValue satisfies the constraint
+/// expressed by this expression. Return true if constraint is not satisfied.
+bool FileCheckNumExpr::VerifyConstraint(
+ FileCheckNumExprVal MatchedValue) const {
+ // Nothing to verify: numeric variable definition with empty numeric
+ // expression.
+ if (AST == nullptr)
+ return false;
+
+ // Will fail if Value or MatchedValue are invalid.
+ return MatchedValue != Value;
+}
+
+/// Store in \p EvaluatedValue the value evaluated from the constraint of this
+/// numeric expression. Return whether evaluation failed.
+bool FileCheckNumExpr::Eval(FileCheckNumExprVal &EvaluatedValue) const {
+ // Nothing to evaluate: numeric variable definition with empty numeric
+ // expression.
+ if (AST == nullptr)
+ return true;
+
+ FileCheckNumExprFmtType ImplicitFmt;
+ EvaluatedValue = AST->Eval(ImplicitFmt);
+ if (!EvaluatedValue.IsValid())
+ return true;
+
+ // If expression comes with an explicit printing format different from the
+ // implicit one, signedness of EvaluatedValue could be different from the
+ // matched value. Convert to signedness of expression to ensure value
+ // compare equally.
+ if (Fmt.Signed)
+ EvaluatedValue.ConvertSigned();
+ else
+ EvaluatedValue.ConvertUnsigned();
+ if (!EvaluatedValue.IsValid())
+ return true;
+
+ return false;
+}
+
+/// Set Value from the result of calling Eval(). Return whether it failed.
+bool FileCheckNumExpr::SetValueFromEval() {
+ // No need to evaluate, we already have a final value.
+ if (Value.IsValid() && !Value.IsTentative())
+ return true;
+
+ FileCheckNumExprVal EvaluatedValue;
+ if (Eval(EvaluatedValue))
+ return true;
+
+ Value = EvaluatedValue;
+ return false;
+}
+
+/// Mark value as final. Return whether value was already final.
+bool FileCheckNumExpr::CommitValue() {
+ // Value is not a valid tentative value.
+ if (!Value.IsValid() || !Value.IsTentative())
+ return true;
+
+ Value.ToggleTentative();
+ return false;
+}
+
+/// Constructor for numeric variable \p Name with a known \p Value at parse
+/// time (eg. the @LINE numeric variable). Matching format of the variable is
+/// given in \p Fmt and \p DefLineNumber indicates when does this variable
+/// starts to be defined.
+FileCheckNumExprVar::FileCheckNumExprVar(StringRef Name,
+ FileCheckNumExprVal Value,
+ struct FileCheckNumExprFmtType Fmt,
+ unsigned DefLineNumber)
+ : Name(Name), DefLineNumber(DefLineNumber) {
+ this->NumExpr = std::make_shared<FileCheckNumExpr>(Value, Fmt);
+}
+
+/// Evaluate the value of this numeric variable. Therefore returns the value
+/// and implicit conversion of this numeric variable stored in its
+/// corresponding numeric expression class.
+FileCheckNumExprVal
+FileCheckNumExprVar::Eval(struct FileCheckNumExprFmtType &Fmt) {
+ // Undefined variable.
+ if (NumExpr == nullptr)
+ return FileCheckNumExprVal();
+ Fmt = NumExpr->GetFormat();
+ return NumExpr->GetValue();
+}
+
+/// Append numeric variable's name to UndefVarNames if undefined.
+void FileCheckNumExprVar::GetUndefVarNames(
+ std::vector<StringRef> &UndefVarNames) const {
+ if (NumExpr == nullptr)
+ UndefVarNames.emplace_back(Name);
+}
+
+/// Evaluate the value of the binary operation represented by this AST. Uses
+/// EvalBinop to perform the binary operation on the values of recursively
+/// evaluating the left and right operands.
+FileCheckNumExprVal
+FileCheckASTBinop::Eval(struct FileCheckNumExprFmtType &Fmt) {
+ struct FileCheckNumExprFmtType LFmt, RFmt;
+ FileCheckNumExprVal LVal = Opl->Eval(LFmt);
+ FileCheckNumExprVal RVal = Opr->Eval(RFmt);
+
+ // Integer promotion.
+ if (!LVal.IsSigned() || !RVal.IsSigned()) {
+ LVal.ConvertUnsigned();
+ RVal.ConvertUnsigned();
+ }
+
+ if (!LVal.IsValid() || !RVal.IsValid())
+ return FileCheckNumExprVal();
+
+ Fmt = (LFmt.Set) ? LFmt : RFmt;
+ if (LFmt.Set && RFmt.Set && LFmt != RFmt)
+ Fmt.Conflict = 1;
+
+ FileCheckNumExprVal Val = EvalBinop(LVal, RVal);
+
+ // Mark result as tentative if any of the operand was tentative. Assert
+ // result was not tentative beforehand to have that check only here.
+ assert(!Val.IsTentative() && "Binary operation returned tentative value");
+ if (LVal.IsTentative() || RVal.IsTentative())
+ Val.ToggleTentative();
+ return Val;
+}
+
+/// Append to UnderVarNames the names of undefined numeric variables used in
+/// any of the operands.
+void FileCheckASTBinop::GetUndefVarNames(
+ std::vector<StringRef> &UndefVarNames) const {
+ Opl->GetUndefVarNames(UndefVarNames);
+ Opr->GetUndefVarNames(UndefVarNames);
+}
+
+/// Perform in \p SubstValue the substitution represented by this class
+/// instance. For numeric variable we replace it by its value if known at
+/// match time or a suitable wildcard pattern otherwise. For pattern variable
+/// we simply replace it by the text its definition matched. Return whether
+/// substitution failed.
+bool FileCheckPatternSubst::Substitute(std::string &SubstValue) const {
+ if (isNumExpr) {
+ if (NumExpr->GetKnownPhase() == Check::MatchPhase) {
+ if (NumExpr->SetValueFromEval())
+ return true;
+ }
+ if (NumExpr->GetMatchString(Check::MatchPhase, SubstValue))
+ return true;
+ return false;
+ } else {
+ // Look up the value and escape it so that we can put it into the
+ // regex.
+ StringRef VarVal;
+ if (Context->GetPatternVarValue(SubstStr, VarVal))
+ return true;
+ SubstValue = Regex::escape(VarVal);
+ }
+
+ return false;
+}
+
+/// Write in \p MatchedValue the value successfully matched (ie. constraint is
+/// verified for a numeric expression) by the substitution. Return whether
+/// match failed for that substitution.
+bool FileCheckPatternSubst::MatchedValue(std::string &MatchedValue) const {
+ if (isNumExpr) {
+ // This will return true if NumExpr's value is invalid, which it will be if
+ // NumExpr did not match or the matched value did not satisfy the
+ // expression constraint.
+ return NumExpr->GetValueStringRepr(MatchedValue);
+ } else {
+ // Look up the value and escape it so that we can put it into the
+ // regex.
+ StringRef VarVal;
+ if (Context->GetPatternVarValue(SubstStr, VarVal))
+ return true;
+ MatchedValue = Regex::escape(VarVal);
+ }
+
+ return false;
+}
+
+/// Append to UnderVarNames the names of undefined pattern or numeric variables
+/// used in this substitution.
+void FileCheckPatternSubst::GetUndefVarNames(
+ std::vector<StringRef> &UndefVarNames) const {
+ if (isNumExpr)
+ NumExpr->GetAST()->GetUndefVarNames(UndefVarNames);
+ else {
+ StringRef VarVal;
+ if (Context->GetPatternVarValue(SubstStr, VarVal))
+ UndefVarNames.emplace_back(SubstStr);
+ }
+}
+
+/// Parsing helper function that strips all leading whitespace from \p s.
+static inline void SkipWhitespace(StringRef &s) { s = s.ltrim(" \t"); }
+
+/// Parsing helper function that strips the string in \p SkipStr from \p s.
+/// Returns true if string \p SkipStr was not in \p s and \p Optional was
+/// false. Returns false otherwise.
+static bool Skip(StringRef &s, const StringRef &SkipStr, bool Optional) {
+ if (!s.consume_front(SkipStr) && !Optional)
+ return true;
+ return false;
+}
+
+/// Parsing helper function that strips the first character in \p s and returns
+/// it.
+static char Next(StringRef &s) {
+ char c = s.front();
+ s = s.drop_front();
+ return c;
+}
+
+// Verify that the string in \p Str or at the start of \p Str (if \p
+// AllowTrailer is true) is a well formed variable name. Also set IsPseudo to
+// true if it is a pseudo variable.
+bool FileCheckPattern::ParseVariable(StringRef Str, bool &IsPseudo,
+ unsigned &TrailIdx) const {
+ bool ParsedOneChar = false;
+ unsigned i = 0;
+ IsPseudo = false;
+ for (unsigned e = Str.size(); i != e; ++i) {
+ if (i == 0) {
+ // Global vars start with '$'.
+ if (Str[i] == '$')
+ continue;
+ else if (Str[i] == '@') {
+ IsPseudo = true;
+ continue;
+ }
+ }
+ // Variable names are composed of alphanumeric characters and underscores.
+ if (Str[i] != '_' && !isalnum(Str[i]))
+ break;
+ ParsedOneChar = true;
+ }
+
+ // Empty name.
+ if (!ParsedOneChar)
+ return true;
+
+ TrailIdx = i;
+ StringRef Name = Str.substr(0, i);
+
+ // Name can't start with a digit.
+ if (isdigit(static_cast<unsigned char>(Name[0])))
+ return true;
+
+ return false;
+}
+
+/// Parse \p Expr for use or definition of a numeric variable. Return the
+/// class instance representing the corresponding variable definition to be
+/// used in the numeric expression AST in the case of a use, or nullptr if
+/// parsing fails in which case errors are reported on \p SM. If
+/// \p IsDefinition is true, a new instance is created. Likewise in case of
+/// use if the variable is undefined at that point in the pattern.
+std::shared_ptr<FileCheckNumExprVar>
+FileCheckPattern::ParseNumericVariable(StringRef &Expr, bool IsDefinition,
+ const SourceMgr &SM) const {
+ bool IsPseudo;
+ unsigned TrailIdx;
+
+ if (ParseVariable(Expr, IsPseudo, TrailIdx))
+ return nullptr;
+ StringRef Name = Expr.substr(0, TrailIdx);
+ Expr = Expr.substr(TrailIdx);
+
+ if (IsPseudo && !Name.equals("@LINE")) {
+ SM.PrintMessage(SMLoc::getFromPointer(Name.data()), SourceMgr::DK_Error,
+ "Unsupported pseudo variable '" + Name + "'");
+ return nullptr;
+ }
+ if (IsDefinition && IsPseudo) {
+ SM.PrintMessage(SMLoc::getFromPointer(Name.data()), SourceMgr::DK_Error,
+ "Definition of pseudo variable '" + Name + "' unsupported");
+ return nullptr;
+ }
+
+ if (IsDefinition) {
+ // Detect collision between pattern and numeric variable when the latter
+ // is created on a later line than the former.
+ if (Context->DefinedVariableTable.find(Name) !=
+ Context->DefinedVariableTable.end()) {
+ SM.PrintMessage(SMLoc::getFromPointer(Name.data()), SourceMgr::DK_Error,
+ "Pattern variable with same name '" + Name + "' exists");
+ return nullptr;
+ }
+ return std::make_shared<FileCheckNumExprVar>(Name, this->LineNumber);
+ } else {
+ // This method is indirectly called from ParsePattern for all numeric
+ // variable definition and uses in the order in which they appear in the
+ // CHECK pattern. For each definition, the pointer to the corresponding
+ // AST class instance is stored in GlobalNumericVariableTable. Therefore
+ // the pointer we get below is for the AST class instance corresponding to
+ // the last definition of the variable before this use.
+ auto git = Context->GlobalNumericVariableTable.find(Name);
+ if (git != Context->GlobalNumericVariableTable.end())
+ return git->second;
+
+ SM.PrintMessage(SMLoc::getFromPointer(Name.data()), SourceMgr::DK_Error,
+ "Using undefined numeric variable '" + Name + "'");
+ return std::make_shared<FileCheckNumExprVar>(Name, 0);
+ }
+}
+
+/// Parse \p Expr for use of a numeric operand. Accept both literal values
+/// and numeric variables. Return the class representing that operand in the
+/// AST of the numeric expression or nullptr if parsing fails in which case
+/// errors are reported on \p SM. Also return in \p ImplicitFmt the implicit
+/// matching format of the operand and in \p KnownPhase the FileCheck phase
+/// when the value of that operand is known.
+std::shared_ptr<FileCheckNumExprAST> FileCheckPattern::ParseNumericOperand(
+ StringRef &Expr, const SourceMgr &SM,
+ struct FileCheckNumExprFmtType &ImplicitFmt,
+ enum Check::FileCheckPhase &KnownPhase) const {
+
+ // Try to parse as a numeric variable use.
+ std::shared_ptr<FileCheckNumExprVar> NumVar =
+ ParseNumericVariable(Expr, false, SM);
+ if (NumVar != nullptr) {
+ FileCheckNumExpr *NumExpr = NumVar->GetNumExpr();
+ // Variable is not undefined.
+ if (NumExpr != nullptr) {
+ ImplicitFmt = NumExpr->GetFormat();
+ if (NumExpr->GetKnownPhase() == Check::ParsePhase)
+ KnownPhase = Check::ParsePhase;
+ // Variable defined on earlier line. Its value will thus be known when
+ // matching this use.
+ else if (NumVar->GetDefLineNumber() < LineNumber)
+ KnownPhase = Check::MatchPhase;
+ else
+ KnownPhase = Check::CheckPhase;
+ }
+ return NumVar;
+ }
+
+ // Otherwise, parse it as a literal.
+ int64_t SignedLiteralValue;
+ uint64_t UnsignedLiteralValue;
+ StringRef SaveExpr = Expr;
+ // Accept both signed and unsigned literal.
+ if (!Expr.consumeInteger(0, SignedLiteralValue)) {
+ ImplicitFmt = FmtNone;
+ KnownPhase = Check::ParsePhase;
+ return std::make_shared<FileCheckNumExprLiteral>(SignedLiteralValue);
+ } else {
+ Expr = SaveExpr;
+ if (!Expr.consumeInteger(0, UnsignedLiteralValue)) {
+ ImplicitFmt = FmtNone;
+ KnownPhase = Check::ParsePhase;
+ return std::make_shared<FileCheckNumExprLiteral>(UnsignedLiteralValue);
+ } else
+ return nullptr;
+ }
+}
+
+/// Parse \p Expr for a binary operation. The left operand of the operand is
+/// given in \p Opl. Return the class representing that binary operation in
+/// the AST of the numeric expression or nullptr if parsing in which case
+/// errors are reported on \p SM. Also return in \p ImplicitFmt the implicit
+/// matching format of the binary operation and in \p KnownPhase the FileCheck
+/// phase when the value of that operand is known.
+std::shared_ptr<FileCheckNumExprAST> FileCheckPattern::ParseFileCheckBinop(
+ StringRef &Expr, std::shared_ptr<FileCheckNumExprAST> Opl,
+ const SourceMgr &SM, struct FileCheckNumExprFmtType &ImplicitFmt,
+ enum Check::FileCheckPhase &KnownPhase) const {
+ SkipWhitespace(Expr);
+ if (Expr.empty())
+ return Opl;
+
+ // Check if this is a supported operation and selection function to perform
+ // it.
+ SMLoc oploc = SMLoc::getFromPointer(Expr.data());
+ char Operator = Next(Expr);
+ binop_eval_t EvalBinop;
+ switch (Operator) {
+ case '+':
+ EvalBinop = FileCheckNumExprVal::Add;
+ break;
+ case '-':
+ EvalBinop = FileCheckNumExprVal::Sub;
+ break;
+ default:
+ SM.PrintMessage(oploc, SourceMgr::DK_Error,
+ std::string("Unsupported numeric operation '") + Operator +
+ "'");
+ return nullptr;
+ }
+
+ SkipWhitespace(Expr);
+ if (Expr.empty())
+ return nullptr;
+
+ // Parse right operand.
+ struct FileCheckNumExprFmtType OplImplicitFmt = ImplicitFmt;
+ enum Check::FileCheckPhase OplKnownPhase = KnownPhase;
+ std::shared_ptr<FileCheckNumExprAST> Opr =
+ ParseNumericOperand(Expr, SM, ImplicitFmt, KnownPhase);
+ if (Opr == nullptr)
+ return nullptr;
+
+ // Compute implicit matching format of the binary operation and indicate
+ // whether the two operands have conflicting implicit format.
+ if (OplImplicitFmt.Set) {
+ if (ImplicitFmt.Set && OplImplicitFmt != ImplicitFmt)
+ ImplicitFmt.Conflict = 1;
+ else
+ ImplicitFmt = OplImplicitFmt;
+ }
+
+ KnownPhase = OplKnownPhase > KnownPhase ? OplKnownPhase : KnownPhase;
+ return std::make_shared<FileCheckASTBinop>(EvalBinop, Opl, Opr);
+}
+
+/// Parse \p Expr for a numeric expression. Return the class representing the
+/// AST of numeric expression or nullptr if parsing fails in which case errors
+/// are reported on \p SM. Set \p NumVarDef to the pointer to the class
+/// representing the variable defined to this numeric expression if any.
+std::shared_ptr<FileCheckNumExpr> FileCheckPattern::ParseNumericExpression(
+ StringRef Expr, std::shared_ptr<FileCheckNumExprVar> &NumVarDef,
+ const SourceMgr &SM) const {
+ struct FileCheckNumExprFmtType Fmt, ExplicitFmt = FmtNone;
+
+ // Parse format specifier.
+ size_t FmtSpecEnd = Expr.find(',');
+ if (FmtSpecEnd != StringRef::npos) {
+ SkipWhitespace(Expr);
+ if (Skip(Expr, "%", false /*Optional*/)) {
+ SM.PrintMessage(
+ SMLoc::getFromPointer(Expr.data()), SourceMgr::DK_Error,
+ "Invalid matching format specification in numeric expression");
+ return nullptr;
+ }
+
+ // Check for unknown matching format specifier and set matching format in
+ // class instance representing this numeric expression.
+ SMLoc fmtloc = SMLoc::getFromPointer(Expr.data());
+ switch (Next(Expr)) {
+ case 'u':
+ ExplicitFmt = FmtUnsigned;
+ break;
+ case 'd':
+ ExplicitFmt = FmtSigned;
+ break;
+ case 'x':
+ ExplicitFmt = FmtLowHex;
+ break;
+ case 'X':
+ ExplicitFmt = FmtCapHex;
+ break;
+ default:
+ SM.PrintMessage(fmtloc, SourceMgr::DK_Error,
+ "Invalid format specifier in numeric expression");
+ return nullptr;
+ }
+
+ SkipWhitespace(Expr);
+ if (Skip(Expr, ",", false /*Optional*/)) {
+ SM.PrintMessage(
+ SMLoc::getFromPointer(Expr.data()), SourceMgr::DK_Error,
+ "Invalid matching format specification in numeric expression");
+ return nullptr;
+ }
+ }
+
+ // Parse numeric variable definition.
+ NumVarDef.reset();
+ size_t DefEnd = Expr.find(':');
+ if (DefEnd != StringRef::npos) {
+ SkipWhitespace(Expr);
+
+ NumVarDef = ParseNumericVariable(Expr, true /*IsDefinition*/, SM);
+ // Invalid variable definition. Error reporting done in parsing function.
+ if (NumVarDef == nullptr)
+ return nullptr;
+
+ SkipWhitespace(Expr);
+ if (Skip(Expr, ":", false /*Optional*/))
+ return nullptr;
+ SkipWhitespace(Expr);
+ }
+
+ // Parse matching constraint.
+ SkipWhitespace(Expr);
+ if (Skip(Expr, "==", true /*Optional*/))
+ return nullptr;
+
+ // Parse numeric expression itself.
+ std::shared_ptr<FileCheckNumExprAST> NumExprAST;
+ struct FileCheckNumExprFmtType ImplicitFmt = FmtNone;
+ enum Check::FileCheckPhase KnownPhase;
+ if (Expr.empty())
+ KnownPhase = Check::CheckPhase;
+ else {
+ SkipWhitespace(Expr);
+ NumExprAST = ParseNumericOperand(Expr, SM, ImplicitFmt, KnownPhase);
+ while (NumExprAST != nullptr && !Expr.empty())
+ NumExprAST =
+ ParseFileCheckBinop(Expr, NumExprAST, SM, ImplicitFmt, KnownPhase);
+ if (NumExprAST == nullptr)
+ return nullptr;
+ }
+
+ // Select explicit matching format if any, implicit one otherwise. Error out
+ // in case of conflicting implicit format without explicit format.
+ if (ExplicitFmt.Set)
+ Fmt = ExplicitFmt;
+ else if (ImplicitFmt.Conflict) {
+ SM.PrintMessage(
+ SMLoc::getFromPointer(Expr.data()), SourceMgr::DK_Error,
+ "Variables with conflicting format specifier: need an explicit one");
+ return nullptr;
+ } else
+ Fmt = ImplicitFmt;
+
+ auto NumExpr =
+ std::make_shared<FileCheckNumExpr>(NumExprAST, Fmt, KnownPhase);
+ if (NumVarDef != nullptr)
+ NumVarDef->SetNumExpr(NumExpr);
+ return NumExpr;
+}
+
+/// Parse legacy numeric expressions in \p Expr, ie. numeric expressions that
+/// were allowed using the pattern variable syntax before numeric expressions
+/// support was added to FileCheck. Pass \p Name of pseudovariable used in the
+/// expression and pass in \p Trailer the rest of the expression. If parsing
+/// fails, errors are reported on \p SM.
+std::shared_ptr<FileCheckNumExpr>
+FileCheckPattern::ParseLegacyNumericExpression(StringRef Expr, StringRef Name,
+ StringRef Trailer,
+ const SourceMgr &SM) const {
+ if (!Name.equals("@LINE")) {
+ SM.PrintMessage(SMLoc::getFromPointer(Name.data()), SourceMgr::DK_Error,
+ "invalid pseudo variable in named regex");
+ return nullptr;
+ }
+
+ if (!Trailer.empty()) {
+ if (Trailer[0] != '+' && Trailer[0] != '-') {
+ SM.PrintMessage(SMLoc::getFromPointer(Trailer.data()),
+ SourceMgr::DK_Error,
+ "invalid operator for pseudo variable in named regex");
+ return nullptr;
+ }
+ Trailer = Trailer.substr(1);
+
+ int Offset;
+ if (Trailer.getAsInteger(10, Offset)) {
+ SM.PrintMessage(SMLoc::getFromPointer(Trailer.data()),
+ SourceMgr::DK_Error,
+ "invalid offset for pseudo variable in named regex");
+ return nullptr;
+ }
+ }
+
+ std::shared_ptr<FileCheckNumExprVar> NumVarDef;
+ std::shared_ptr<FileCheckNumExpr> NumExpr =
+ ParseNumericExpression(Expr, NumVarDef, SM);
+ assert(NumVarDef == nullptr && "Legacy numeric expression sets variable");
+ return NumExpr;
+}
+
/// Parses the given string into the Pattern.
///
/// \p Prefix provides which prefix is being matched, \p SM provides the
@@ -39,6 +862,14 @@ bool FileCheckPattern::ParsePattern(StringRef PatternStr, StringRef Prefix,
this->LineNumber = LineNumber;
PatternLoc = SMLoc::getFromPointer(PatternStr.data());
+ // Create fake @LINE pseudo variable definition.
+ StringRef LinePseudo = "@LINE";
+ uint64_t LineNumber64 = LineNumber;
+ auto LineNumberVal = FileCheckNumExprVal(LineNumber64);
+ auto LinePseudoVar = std::make_shared<FileCheckNumExprVar>(
+ LinePseudo, LineNumberVal, FmtUnsigned, LineNumber);
+ Context->GlobalNumericVariableTable[LinePseudo] = LinePseudoVar;
+
if (!(Req.NoCanonicalizeWhiteSpace && Req.MatchFullLines))
// Ignore trailing whitespace.
while (!PatternStr.empty() &&
@@ -118,92 +949,215 @@ bool FileCheckPattern::ParsePattern(StringRef PatternStr, StringRef Prefix,
// itself must be of the form "[a-zA-Z_][0-9a-zA-Z_]*", otherwise we reject
// it. This is to catch some common errors.
if (PatternStr.startswith("[[")) {
+ StringRef MatchStr = PatternStr.substr(2);
+ bool IsNumExpr = MatchStr.consume_front("#");
+ const char *RefTypeStr = IsNumExpr ? "numeric expression" : "named regex";
// Find the closing bracket pair ending the match. End is going to be an
// offset relative to the beginning of the match string.
- size_t End = FindRegexVarEnd(PatternStr.substr(2), SM);
+ size_t End = FindRegexVarEnd(MatchStr, SM);
if (End == StringRef::npos) {
- SM.PrintMessage(SMLoc::getFromPointer(PatternStr.data()),
- SourceMgr::DK_Error,
- "invalid named regex reference, no ]] found");
+ SM.PrintMessage(
+ SMLoc::getFromPointer(PatternStr.data()), SourceMgr::DK_Error,
+ std::string("invalid ") + RefTypeStr + " reference, no ]] found");
return true;
}
- StringRef MatchStr = PatternStr.substr(2, End);
- PatternStr = PatternStr.substr(End + 4);
+ MatchStr = MatchStr.substr(0, End);
+ PatternStr = PatternStr.substr(End + 4 + (int)IsNumExpr);
+
+ bool CaptureParenNeeded;
+ bool SubstNeeded;
+ bool MatchSubstNeeded;
+ bool IsVarDef;
+ StringRef DefName;
+ StringRef SubstStr;
+ StringRef MatchRegexp;
+ unsigned SubstInsertIdx = RegExStr.size();
+ std::shared_ptr<FileCheckNumExprVar> NumVarDef;
+ std::shared_ptr<FileCheckNumExpr> NumExpr;
+
+ // Parse pattern variable or legacy numeric expression.
+ if (!IsNumExpr) {
+ // Get the regex name (e.g. "foo") and verify it is well formed.
+ bool IsPseudo;
+ unsigned TrailIdx;
+ if (ParseVariable(MatchStr, IsPseudo, TrailIdx)) {
+ SM.PrintMessage(SMLoc::getFromPointer(MatchStr.data()),
+ SourceMgr::DK_Error, "invalid name in named regex");
+ return true;
+ }
- // Get the regex name (e.g. "foo").
- size_t NameEnd = MatchStr.find(':');
- StringRef Name = MatchStr.substr(0, NameEnd);
+ StringRef Name = MatchStr.substr(0, TrailIdx);
+ StringRef Trailer = MatchStr.substr(TrailIdx);
+ size_t DefSepIdx = Trailer.find(":");
+ IsVarDef = (DefSepIdx != StringRef::npos);
+ MatchSubstNeeded = !IsVarDef;
+ SubstNeeded = MatchSubstNeeded;
+ CaptureParenNeeded = IsVarDef;
- if (Name.empty()) {
- SM.PrintMessage(SMLoc::getFromPointer(Name.data()), SourceMgr::DK_Error,
- "invalid name in named regex: empty name");
- return true;
- }
+ if (IsVarDef && (IsPseudo || !Trailer.consume_front(":"))) {
+ SM.PrintMessage(SMLoc::getFromPointer(MatchStr.data()),
+ SourceMgr::DK_Error,
+ "invalid name in named regex definition");
+ return true;
+ }
- // Verify that the name/expression is well formed. FileCheck currently
- // supports @LINE, @LINE+number, @LINE-number expressions. The check here
- // is relaxed, more strict check is performed in \c EvaluateExpression.
- bool IsExpression = false;
- for (unsigned i = 0, e = Name.size(); i != e; ++i) {
- if (i == 0) {
- if (Name[i] == '$') // Global vars start with '$'
- continue;
- if (Name[i] == '@') {
- if (NameEnd != StringRef::npos) {
- SM.PrintMessage(SMLoc::getFromPointer(Name.data()),
- SourceMgr::DK_Error,
- "invalid name in named regex definition");
+ if (IsVarDef) {
+ // Detect collision between pattern and numeric variable when the
+ // former is created on a later line than the latter.
+ if (Context->GlobalNumericVariableTable.find(Name) !=
+ Context->GlobalNumericVariableTable.end()) {
+ SM.PrintMessage(
+ SMLoc::getFromPointer(MatchStr.data()), SourceMgr::DK_Error,
+ "Numeric variable with same name '" + Name + "' exists");
+ return true;
+ }
+ DefName = Name;
+ MatchRegexp = Trailer;
+ } else {
+ SubstStr = MatchStr;
+
+ // Syntax for pattern variables allowed for simple expressions based
+ // on @LINE pseudo variable (aka legacy numeric expressions) before
+ // numeric expression support was added. Accept those (and only
+ // those) for compatibility.
+ if (IsPseudo) {
+ NumExpr = ParseLegacyNumericExpression(MatchStr, Name, Trailer, SM);
+ if (NumExpr == nullptr)
return true;
- }
- IsExpression = true;
- continue;
+ IsNumExpr = true;
+ } else if (!Trailer.empty()) {
+ SM.PrintMessage(SMLoc::getFromPointer(Trailer.data()),
+ SourceMgr::DK_Error,
+ "invalid use of operator in named regex");
+ return true;
}
}
- if (Name[i] != '_' && !isalnum(Name[i]) &&
- (!IsExpression || (Name[i] != '+' && Name[i] != '-'))) {
- SM.PrintMessage(SMLoc::getFromPointer(Name.data() + i),
- SourceMgr::DK_Error, "invalid name in named regex");
+ // Parse numeric expression.
+ } else {
+ SubstStr = MatchStr;
+ NumExpr = ParseNumericExpression(SubstStr, NumVarDef, SM);
+ if (NumExpr == nullptr)
+ return true;
+ }
+
+ // Process numeric expressions and legacy numeric expressions.
+ if (IsNumExpr) {
+ IsVarDef = (NumVarDef != nullptr);
+ if (IsVarDef)
+ DefName = NumVarDef->GetName();
+
+ // Evaluate the numeric expression if we can already do it so that we
+ // directly add it to the pattern to match and don't have to do a
+ // substitution later.
+ if (NumExpr->GetKnownPhase() == Check::ParsePhase &&
+ NumExpr->SetValueFromEval()) {
+ SM.PrintMessage(
+ SMLoc::getFromPointer(SubstStr.data()), SourceMgr::DK_Error,
+ "Failed to evaluate numeric expression known at parse time");
return true;
}
+
+ // Expression can be evaluated at match time, it will need a
+ // substitution.
+ MatchSubstNeeded = (NumExpr->GetKnownPhase() == Check::MatchPhase);
+ SubstNeeded = (NumExpr->GetAST() != nullptr);
+
+ // Expression will only be known after matching is done, matched value
+ // needs to be captured and verified against the constraint of the
+ // numeric expression.
+ if (NumExpr->GetKnownPhase() > Check::MatchPhase) {
+ CaptureParenNeeded = true;
+ FileCheckNumExprMatch NumExprMatch = {NumExpr, CurParen};
+ CapturedNumericExpressions.emplace_back(NumExprMatch);
+ // No capture needed otherwise.
+ } else
+ CaptureParenNeeded = false;
+
+ // Get the string we need to match if we know it already. This could
+ // be either the value if we already know it or a suitable wildcard
+ // pattern if value would be known after match and needs to be verified
+ // against the constraint of the numeric expression. Numeric
+ // expression whose evaluation is known at match time get an empty
+ // string and a substitution is created instead.
+ std::string MatchString;
+ if (NumExpr->GetMatchString(Check::ParsePhase, MatchString))
+ assert(false && "Failed to get string to match");
+ MatchRegexp = StringRef(MatchString);
}
- // Name can't start with a digit.
- if (isdigit(static_cast<unsigned char>(Name[0]))) {
- SM.PrintMessage(SMLoc::getFromPointer(Name.data()), SourceMgr::DK_Error,
- "invalid name in named regex");
- return true;
+ // Capturing parentheses are needed. Numeric expressions using variable
+ // defined on the same line need it to do the match in a single pass:
+ // they are matched using an appropriate wildcard pattern and the matched
+ // value gets verified against the constraint of the numeric expression
+ // later.
+ if (CaptureParenNeeded) {
+ RegExStr += '(';
+ // Checking numeric expressions requires substituting them by suitable
+ // wildcard patterns and capturing what these matched to. Adjust
+ // substitution index to be inside the capturing parentheses.
+ ++SubstInsertIdx;
}
- // Handle [[foo]].
- if (NameEnd == StringRef::npos) {
- // Handle variables that were defined earlier on the same line by
- // emitting a backreference.
- if (VariableDefs.find(Name) != VariableDefs.end()) {
- unsigned VarParenNum = VariableDefs[Name];
- if (VarParenNum < 1 || VarParenNum > 9) {
- SM.PrintMessage(SMLoc::getFromPointer(Name.data()),
- SourceMgr::DK_Error,
- "Can't back-reference more than 9 variables");
- return true;
- }
- AddBackrefToRegEx(VarParenNum);
- } else {
- VariableUses.push_back(std::make_pair(Name, RegExStr.size()));
+ // Handle variable definition: [[<def>:(...)]] and [[#(...)<def>:(...)]].
+ if (IsVarDef) {
+ if (IsNumExpr)
+ // This store is done here rather than in Match() to allow
+ // ParseNumericVariable() to get the pointer to the AST class
+ // instance of the right variable definition corresponding to a
+ // given numeric variable use.
+ Context->GlobalNumericVariableTable[DefName] = NumVarDef;
+ else {
+ VariableDefs[DefName] = CurParen;
+ // Mark pattern variable as defined to detect collision between
+ // pattern and numeric variable in ParseNumericVariable when the
+ // latter is created on a later line than the former. We cannot
+ // reuse GlobalVariableTable by populating it with an empty string
+ // for that since we would then loose the ability to detect use of
+ // undefined variable in Match().
+ Context->DefinedVariableTable[DefName] = true;
}
- continue;
}
- // Handle [[foo:.*]].
- VariableDefs[Name] = CurParen;
- RegExStr += '(';
- ++CurParen;
+ if (CaptureParenNeeded)
+ ++CurParen;
- if (AddRegExToRegEx(MatchStr.substr(NameEnd + 1), CurParen, SM))
+ if (!MatchRegexp.empty() && AddRegExToRegEx(MatchRegexp, CurParen, SM))
return true;
- RegExStr += ')';
+ if (CaptureParenNeeded)
+ RegExStr += ')';
+
+ // Handle use of pattern variables that were defined earlier on the
+ // same line by emitting a backreference. Numeric expressions cannot
+ // be expressed using regexs so are recorded as substitutions to
+ // perform by FileCheck in the matching phase when only using variables
+ // defined on the line. See Match() for how numeric expressions using
+ // any variable defined on the same line are matched.
+ if (!IsNumExpr && VariableDefs.find(SubstStr) != VariableDefs.end()) {
+ unsigned CaptureParen = VariableDefs[SubstStr];
+ if (CaptureParen < 1 || CaptureParen > 9) {
+ SM.PrintMessage(SMLoc::getFromPointer(SubstStr.data()),
+ SourceMgr::DK_Error,
+ "Can't back-reference more than 9 variables");
+ return true;
+ }
+ AddBackrefToRegEx(CaptureParen);
+ // Handle use of pattern variables ([[<var>]]) defined in previous CHECK
+ // pattern or use of a numeric expression.
+ } else if (SubstNeeded) {
+ FileCheckPatternSubst Subst =
+ IsNumExpr
+ ? FileCheckPatternSubst(Context, SubstStr, NumExpr,
+ SubstInsertIdx)
+ : FileCheckPatternSubst(Context, SubstStr, SubstInsertIdx);
+ AllSubsts.push_back(Subst);
+ // Handle use of a numeric expression with all variables defined in
+ // previous CHECK pattern.
+ if (MatchSubstNeeded)
+ MatchSubsts.push_back(Subst);
+ }
}
// Handle fixed string matches.
@@ -243,37 +1197,20 @@ void FileCheckPattern::AddBackrefToRegEx(unsigned BackrefNum) {
RegExStr += Backref;
}
-/// Evaluates expression and stores the result to \p Value.
-///
-/// Returns true on success and false when the expression has invalid syntax.
-bool FileCheckPattern::EvaluateExpression(StringRef Expr, std::string &Value) const {
- // The only supported expression is @LINE([\+-]\d+)?
- if (!Expr.startswith("@LINE"))
- return false;
- Expr = Expr.substr(StringRef("@LINE").size());
- int Offset = 0;
- if (!Expr.empty()) {
- if (Expr[0] == '+')
- Expr = Expr.substr(1);
- else if (Expr[0] != '-')
- return false;
- if (Expr.getAsInteger(10, Offset))
- return false;
- }
- Value = llvm::itostr(LineNumber + Offset);
- return true;
-}
-
/// Matches the pattern string against the input buffer \p Buffer
///
/// This returns the position that is matched or npos if there is no match. If
/// there is a match, the size of the matched string is returned in \p
/// MatchLen.
///
-/// The \p VariableTable StringMap provides the current values of filecheck
-/// variables and is updated if this match defines new values.
+/// The GlobalVariableTable StringMap provides the current values of FileCheck
+/// pattern variables and is updated if this match defines new values.
+/// Substitutions for numeric expression point to the class instance
+/// representing the numeric expression which is updated when the value is
+/// known, either in this function or in ParsePattern if their value is known
+/// at parse time.
size_t FileCheckPattern::Match(StringRef Buffer, size_t &MatchLen,
- StringMap<StringRef> &VariableTable) const {
+ const SourceMgr &SM) const {
// If this is the EOF pattern, match it immediately.
if (CheckTy == Check::CheckEOF) {
MatchLen = 0;
@@ -292,29 +1229,43 @@ size_t FileCheckPattern::Match(StringRef Buffer, size_t &MatchLen,
// actual value.
StringRef RegExToMatch = RegExStr;
std::string TmpStr;
- if (!VariableUses.empty()) {
+ SmallVector<StringRef, 4> MatchInfo;
+
+ if (!MatchSubsts.empty()) {
TmpStr = RegExStr;
unsigned InsertOffset = 0;
- for (const auto &VariableUse : VariableUses) {
+ // Substitute all pattern variables and numeric expressions whose value is
+ // known just now. Use of pattern variables defined on the same line are
+ // handled by back-references. Numeric expressions whose value is known at
+ // parse time (eg. uses of @LINE pseudo variable) are replaced at parse
+ // time. For numeric expressions using variable defined on the same line,
+ // RegExStr is filled with a suitable wildcard pattern and the constraint
+ // is then verified against the matched value.
+ //
+ // Known issue: Due to the constraints for numeric expressions using
+ // variable defined in earlier line not being expressed at the regular
+ // expression level, the wrong values might be matched, leading to a
+ // constraint check failure. An example of this is what happens when
+ // checking the line "1 3 4" against the directive
+ // "CHECK: [[#N:]] [[#N+1]]" which gets substituted by
+ // "[[:digit:]]+ [[:digit:]]+". This would match 1 3 which then fails the
+ // constraint checks.
+ //
+ // One possible solution would be to augment the regular expression engine
+ // to be able to return all matches for a given pattern. FileCheck could
+ // then test all possibility and fail the check only if none of them
+ // satisfy all numeric expressions. This would work both for numeric
+ // expression using a numeric variable defined on the same line as well as
+ // numeric expressions with a comparison rather than equality constraint.
+ for (const auto &MatchSubst : MatchSubsts) {
+ // Substitute and check for failure (eg. use of undefined variable).
std::string Value;
-
- if (VariableUse.first[0] == '@') {
- if (!EvaluateExpression(VariableUse.first, Value))
- return StringRef::npos;
- } else {
- StringMap<StringRef>::iterator it =
- VariableTable.find(VariableUse.first);
- // If the variable is undefined, return an error.
- if (it == VariableTable.end())
- return StringRef::npos;
-
- // Look up the value and escape it so that we can put it into the regex.
- Value += Regex::escape(it->second);
- }
+ if (MatchSubst.Substitute(Value))
+ return StringRef::npos;
// Plop it into the regex at the adjusted offset.
- TmpStr.insert(TmpStr.begin() + VariableUse.second + InsertOffset,
+ TmpStr.insert(TmpStr.begin() + MatchSubst.GetIndex() + InsertOffset,
Value.begin(), Value.end());
InsertOffset += Value.size();
}
@@ -323,35 +1274,94 @@ size_t FileCheckPattern::Match(StringRef Buffer, size_t &MatchLen,
RegExToMatch = TmpStr;
}
- SmallVector<StringRef, 4> MatchInfo;
- if (!Regex(RegExToMatch, Regex::Newline).match(Buffer, &MatchInfo))
- return StringRef::npos;
+ bool AllNumExprSatisfied;
+ do {
+ AllNumExprSatisfied = true;
+ if (!Regex(RegExToMatch, Regex::Newline).match(Buffer, &MatchInfo))
+ return StringRef::npos;
- // Successful regex match.
- assert(!MatchInfo.empty() && "Didn't get any match");
- StringRef FullMatch = MatchInfo[0];
+ // Successful regex match.
+ assert(!MatchInfo.empty() && "Didn't get any match");
- // If this defines any variables, remember their values.
- for (const auto &VariableDef : VariableDefs) {
- assert(VariableDef.second < MatchInfo.size() && "Internal paren error");
- VariableTable[VariableDef.first] = MatchInfo[VariableDef.second];
- }
+ // If this defines any pattern variables, remember their values.
+ for (const auto &VariableDef : VariableDefs) {
+ StringRef Name = VariableDef.first;
+ unsigned CaptureParen = VariableDef.second;
+ assert(CaptureParen < MatchInfo.size() && "Internal paren error");
+ StringRef MatchedStr = MatchInfo[CaptureParen];
+ Context->GlobalVariableTable[Name] = MatchedStr;
+ }
+
+ // Check the matched value satisfies the numeric expression constraint and
+ // store it in the numeric expression if that's the case.
+ for (const auto &CapturedNumericExpression : CapturedNumericExpressions) {
+ assert(CapturedNumericExpression.CaptureParen < MatchInfo.size() &&
+ "Internal paren error");
+ StringRef MatchedStr = MatchInfo[CapturedNumericExpression.CaptureParen];
+ FileCheckNumExpr *NumExpr = CapturedNumericExpression.NumExpr.get();
+
+ // If numeric expression can be evaluated now, do it and error out if
+ // evaluation fails.
+ bool UnconstrainedVarDef = (NumExpr->GetAST() == nullptr);
+ if (!UnconstrainedVarDef) {
+ if (NumExpr->SetValueFromEval())
+ return StringRef::npos;
+ }
+
+ // Get numeric value from the matched string according to matching format
+ // of the numeric expression. Error out if this fails.
+ FileCheckNumExprVal MatchedVal;
+ if (NumExpr->ValueFromStringRepr(MatchedStr, MatchedVal)) {
+ SM.PrintMessage(SMLoc::getFromPointer(MatchedStr.data()),
+ SourceMgr::DK_Error,
+ "Cannot represent numeric value " + MatchedStr);
+ return StringRef::npos;
+ }
+
+ // Now verify the constraint of the numeric expression.
+ if (NumExpr->VerifyConstraint(MatchedVal)) {
+ AllNumExprSatisfied = false;
+ // Try matching again from the line following the one matched. There
+ // might be over match in the current line but we have no way to ask
+ // for them to the regular expression engine. However this hazard can
+ // be alleviated if the user write a CHECK pattern that has numeric
+ // expression for all numeric values the line to match contains.
+ Buffer = Buffer.substr(Buffer.find_first_of('\n')).substr(1);
+ break;
+ }
+
+ // Store verified matched value. This will set variable defined by the
+ // expression if any.
+ if (UnconstrainedVarDef) {
+ if (NumExpr->SetTentativeValue(MatchedVal))
+ assert(false && "Value of numeric expression already set");
+ }
+ }
+ } while (!AllNumExprSatisfied && !Buffer.empty());
+
+ // All constraints satisfied, make variable definitions final.
+ if (AllNumExprSatisfied) {
+ for (const auto &CapturedNumericExpression : CapturedNumericExpressions) {
+ FileCheckNumExpr *NumExpr = CapturedNumericExpression.NumExpr.get();
+ NumExpr->CommitValue();
+ }
+ // No line with all constraints satisfied found, return failure to match.
+ } else
+ return StringRef::npos;
// Like CHECK-NEXT, CHECK-EMPTY's match range is considered to start after
// the required preceding newline, which is consumed by the pattern in the
// case of CHECK-EMPTY but not CHECK-NEXT.
size_t MatchStartSkip = CheckTy == Check::CheckEmpty;
+ StringRef FullMatch = MatchInfo[0];
MatchLen = FullMatch.size() - MatchStartSkip;
return FullMatch.data() - Buffer.data() + MatchStartSkip;
}
-
/// Computes an arbitrary estimate for the quality of matching this pattern at
/// the start of \p Buffer; a distance of zero should correspond to a perfect
/// match.
-unsigned
-FileCheckPattern::ComputeMatchDistance(StringRef Buffer,
- const StringMap<StringRef> &VariableTable) const {
+unsigned FileCheckPattern::ComputeMatchDistance(StringRef Buffer) const {
// Just compute the number of matching characters. For regular expressions, we
// just compare against the regex itself and hope for the best.
//
@@ -368,38 +1378,50 @@ FileCheckPattern::ComputeMatchDistance(StringRef Buffer,
return BufferPrefix.edit_distance(ExampleString);
}
-void FileCheckPattern::PrintVariableUses(const SourceMgr &SM, StringRef Buffer,
- const StringMap<StringRef> &VariableTable,
- SMRange MatchRange) const {
- // If this was a regular expression using variables, print the current
- // variable values.
- if (!VariableUses.empty()) {
- for (const auto &VariableUse : VariableUses) {
+/// Print value of successful substitutions or name of undefined pattern or
+/// numeric variables preventing such a successful substitution.
+///
+/// Note: Numeric expressions known at parse time are not shown here since no
+/// substitution gets created for them but hopefully if the value is known at
+/// parse time the value should be obvious (except in case of bug in FileCheck
+/// itself). Numeric expression using numeric variables defined on the same
+/// line are not shown either.
+void FileCheckPattern::PrintSubsts(const SourceMgr &SM, StringRef Buffer,
+ SMRange MatchRange) const {
+ // Print info what we know about substitutions. This covers both uses of
+ // pattern variables and numeric subsitutions as long as they only use
+ // variables defined in earlier lines.
+ if (!AllSubsts.empty()) {
+ for (const auto &Subst : AllSubsts) {
SmallString<256> Msg;
raw_svector_ostream OS(Msg);
- StringRef Var = VariableUse.first;
- if (Var[0] == '@') {
- std::string Value;
- if (EvaluateExpression(Var, Value)) {
- OS << "with expression \"";
- OS.write_escaped(Var) << "\" equal to \"";
- OS.write_escaped(Value) << "\"";
- } else {
- OS << "uses incorrect expression \"";
- OS.write_escaped(Var) << "\"";
+ bool IsNumExpr = Subst.IsNumExpr();
+ std::string Val;
+
+ // Substitution failed or is not known at match time, print undefined
+ // variables it uses.
+ if (Subst.MatchedValue(Val)) {
+ std::vector<StringRef> UndefVarNames;
+ Subst.GetUndefVarNames(UndefVarNames);
+ if (UndefVarNames.empty())
+ continue;
+ if (IsNumExpr)
+ OS << "uses undefined numeric variables";
+ else
+ OS << "uses undefined variable";
+ for (auto UndefVarName : UndefVarNames) {
+ OS << " \"";
+ OS.write_escaped(UndefVarName) << "\"";
}
+ // Substitution succeeded and numeric expression satisfied its
+ // constraint. Print substituted value.
} else {
- StringMap<StringRef>::const_iterator it = VariableTable.find(Var);
-
- // Check for undefined variable references.
- if (it == VariableTable.end()) {
- OS << "uses undefined variable \"";
- OS.write_escaped(Var) << "\"";
- } else {
+ if (IsNumExpr)
+ OS << "with numeric expression \"";
+ else
OS << "with variable \"";
- OS.write_escaped(Var) << "\" equal to \"";
- OS.write_escaped(it->second) << "\"";
- }
+ OS.write_escaped(Subst.GetSubstString()) << "\" equal to \"";
+ OS.write_escaped(Val) << "\"";
}
if (MatchRange.isValid())
@@ -432,7 +1454,6 @@ static SMRange ProcessMatchResult(FileCheckDiag::MatchType MatchTy,
void FileCheckPattern::PrintFuzzyMatch(
const SourceMgr &SM, StringRef Buffer,
- const StringMap<StringRef> &VariableTable,
std::vector<FileCheckDiag> *Diags) const {
// Attempt to find the closest/best fuzzy match. Usually an error happens
// because some string in the output didn't exactly match. In these cases, we
@@ -454,7 +1475,7 @@ void FileCheckPattern::PrintFuzzyMatch(
// Compute the "quality" of this match as an arbitrary combination of the
// match distance and the number of lines skipped to get to this match.
- unsigned Distance = ComputeMatchDistance(Buffer.substr(i), VariableTable);
+ unsigned Distance = ComputeMatchDistance(Buffer.substr(i));
double Quality = Distance + (NumLinesForward / 100.);
if (Quality < BestQuality || Best == StringRef::npos) {
@@ -478,6 +1499,18 @@ void FileCheckPattern::PrintFuzzyMatch(
}
}
+/// Store in \p VarValue the matched value for pattern variable \p VarName if
+/// defined. Return whether \p VarName is undefined.
+bool FileCheckPatternContext::GetPatternVarValue(StringRef VarName,
+ StringRef &VarValue) {
+ auto git = GlobalVariableTable.find(VarName);
+ if (git == GlobalVariableTable.end())
+ return true;
+
+ VarValue = git->second;
+ return false;
+}
+
/// Finds the closing sequence of a regex variable usage or definition.
///
/// \p Str has to point in the beginning of the definition (right after the
@@ -748,9 +1781,15 @@ FindFirstMatchingPrefix(Regex &PrefixRE, StringRef &Buffer,
///
/// The strings are added to the CheckStrings vector. Returns true in case of
/// an error, false otherwise.
-bool llvm::FileCheck::ReadCheckFile(SourceMgr &SM, StringRef Buffer,
- Regex &PrefixRE,
- std::vector<FileCheckString> &CheckStrings) {
+bool llvm::FileCheck::ReadCheckFile(
+ SourceMgr &SM, StringRef Buffer, Regex &PrefixRE,
+ std::vector<FileCheckString> &CheckStrings) {
+ auto *PatternContext = new FileCheckPatternContext();
+ // Process command-line definition of variables at parse time so that numeric
+ // expression can refer to numeric variable defined on the command-line.
+ if (PatternContext->DefineCmdlineVariables(Req.GlobalDefines, SM))
+ return true;
+
std::vector<FileCheckPattern> ImplicitNegativeChecks;
for (const auto &PatternString : Req.ImplicitCheckNot) {
// Create a buffer with fake command line content in order to display the
@@ -764,7 +1803,8 @@ bool llvm::FileCheck::ReadCheckFile(SourceMgr &SM, StringRef Buffer,
CmdLine->getBuffer().substr(Prefix.size(), PatternString.size());
SM.AddNewSourceBuffer(std::move(CmdLine), SMLoc());
- ImplicitNegativeChecks.push_back(FileCheckPattern(Check::CheckNot));
+ ImplicitNegativeChecks.push_back(
+ FileCheckPattern(Check::CheckNot, PatternContext));
ImplicitNegativeChecks.back().ParsePattern(PatternInBuffer,
"IMPLICIT-CHECK", SM, 0, Req);
}
@@ -827,7 +1867,7 @@ bool llvm::FileCheck::ReadCheckFile(SourceMgr &SM, StringRef Buffer,
SMLoc PatternLoc = SMLoc::getFromPointer(Buffer.data());
// Parse the pattern.
- FileCheckPattern P(CheckTy);
+ FileCheckPattern P(CheckTy, PatternContext);
if (P.ParsePattern(Buffer.substr(0, EOL), UsedPrefix, SM, LineNumber, Req))
return true;
@@ -871,7 +1911,8 @@ bool llvm::FileCheck::ReadCheckFile(SourceMgr &SM, StringRef Buffer,
// Add an EOF pattern for any trailing CHECK-DAG/-NOTs, and use the first
// prefix as a filler for the error message.
if (!DagNotMatches.empty()) {
- CheckStrings.emplace_back(FileCheckPattern(Check::CheckEOF), *Req.CheckPrefixes.begin(),
+ CheckStrings.emplace_back(FileCheckPattern(Check::CheckEOF, PatternContext),
+ *Req.CheckPrefixes.begin(),
SMLoc::getFromPointer(Buffer.data()));
std::swap(DagNotMatches, CheckStrings.back().DagNotStrings);
}
@@ -897,8 +1938,7 @@ bool llvm::FileCheck::ReadCheckFile(SourceMgr &SM, StringRef Buffer,
static void PrintMatch(bool ExpectedMatch, const SourceMgr &SM,
StringRef Prefix, SMLoc Loc, const FileCheckPattern &Pat,
- int MatchedCount, StringRef Buffer,
- StringMap<StringRef> &VariableTable, size_t MatchPos,
+ int MatchedCount, StringRef Buffer, size_t MatchPos,
size_t MatchLen, const FileCheckRequest &Req,
std::vector<FileCheckDiag> *Diags) {
if (ExpectedMatch) {
@@ -922,24 +1962,22 @@ static void PrintMatch(bool ExpectedMatch, const SourceMgr &SM,
Loc, ExpectedMatch ? SourceMgr::DK_Remark : SourceMgr::DK_Error, Message);
SM.PrintMessage(MatchRange.Start, SourceMgr::DK_Note, "found here",
{MatchRange});
- Pat.PrintVariableUses(SM, Buffer, VariableTable, MatchRange);
+ Pat.PrintSubsts(SM, Buffer, MatchRange);
}
static void PrintMatch(bool ExpectedMatch, const SourceMgr &SM,
const FileCheckString &CheckStr, int MatchedCount,
- StringRef Buffer, StringMap<StringRef> &VariableTable,
- size_t MatchPos, size_t MatchLen, FileCheckRequest &Req,
+ StringRef Buffer, size_t MatchPos, size_t MatchLen,
+ FileCheckRequest &Req,
std::vector<FileCheckDiag> *Diags) {
PrintMatch(ExpectedMatch, SM, CheckStr.Prefix, CheckStr.Loc, CheckStr.Pat,
- MatchedCount, Buffer, VariableTable, MatchPos, MatchLen, Req,
- Diags);
+ MatchedCount, Buffer, MatchPos, MatchLen, Req, Diags);
}
static void PrintNoMatch(bool ExpectedMatch, const SourceMgr &SM,
StringRef Prefix, SMLoc Loc,
const FileCheckPattern &Pat, int MatchedCount,
- StringRef Buffer, StringMap<StringRef> &VariableTable,
- bool VerboseVerbose,
+ StringRef Buffer, bool VerboseVerbose,
std::vector<FileCheckDiag> *Diags) {
if (!ExpectedMatch && !VerboseVerbose)
return;
@@ -965,19 +2003,18 @@ static void PrintNoMatch(bool ExpectedMatch, const SourceMgr &SM,
SM.PrintMessage(SearchRange.Start, SourceMgr::DK_Note, "scanning from here");
// Allow the pattern to print additional information if desired.
- Pat.PrintVariableUses(SM, Buffer, VariableTable);
+ Pat.PrintSubsts(SM, Buffer);
if (ExpectedMatch)
- Pat.PrintFuzzyMatch(SM, Buffer, VariableTable, Diags);
+ Pat.PrintFuzzyMatch(SM, Buffer, Diags);
}
static void PrintNoMatch(bool ExpectedMatch, const SourceMgr &SM,
const FileCheckString &CheckStr, int MatchedCount,
- StringRef Buffer, StringMap<StringRef> &VariableTable,
- bool VerboseVerbose,
+ StringRef Buffer, bool VerboseVerbose,
std::vector<FileCheckDiag> *Diags) {
PrintNoMatch(ExpectedMatch, SM, CheckStr.Prefix, CheckStr.Loc, CheckStr.Pat,
- MatchedCount, Buffer, VariableTable, VerboseVerbose, Diags);
+ MatchedCount, Buffer, VerboseVerbose, Diags);
}
/// Count the number of newlines in the specified range.
@@ -1006,7 +2043,6 @@ static unsigned CountNumNewlinesBetween(StringRef Range,
/// Match check string and its "not strings" and/or "dag strings".
size_t FileCheckString::Check(const SourceMgr &SM, StringRef Buffer,
bool IsLabelScanMode, size_t &MatchLen,
- StringMap<StringRef> &VariableTable,
FileCheckRequest &Req,
std::vector<FileCheckDiag> *Diags) const {
size_t LastPos = 0;
@@ -1018,7 +2054,7 @@ size_t FileCheckString::Check(const SourceMgr &SM, StringRef Buffer,
// over the block again (including the last CHECK-LABEL) in normal mode.
if (!IsLabelScanMode) {
// Match "dag strings" (with mixed "not strings" if any).
- LastPos = CheckDag(SM, Buffer, NotStrings, VariableTable, Req, Diags);
+ LastPos = CheckDag(SM, Buffer, NotStrings, Req, Diags);
if (LastPos == StringRef::npos)
return StringRef::npos;
}
@@ -1033,18 +2069,17 @@ size_t FileCheckString::Check(const SourceMgr &SM, StringRef Buffer,
StringRef MatchBuffer = Buffer.substr(LastMatchEnd);
size_t CurrentMatchLen;
// get a match at current start point
- size_t MatchPos = Pat.Match(MatchBuffer, CurrentMatchLen, VariableTable);
+ size_t MatchPos = Pat.Match(MatchBuffer, CurrentMatchLen, SM);
if (i == 1)
FirstMatchPos = LastPos + MatchPos;
// report
if (MatchPos == StringRef::npos) {
- PrintNoMatch(true, SM, *this, i, MatchBuffer, VariableTable,
- Req.VerboseVerbose, Diags);
+ PrintNoMatch(true, SM, *this, i, MatchBuffer, Req.VerboseVerbose, Diags);
return StringRef::npos;
}
- PrintMatch(true, SM, *this, i, MatchBuffer, VariableTable, MatchPos,
- CurrentMatchLen, Req, Diags);
+ PrintMatch(true, SM, *this, i, MatchBuffer, MatchPos, CurrentMatchLen, Req,
+ Diags);
// move start point after the match
LastMatchEnd += MatchPos + CurrentMatchLen;
@@ -1079,7 +2114,7 @@ size_t FileCheckString::Check(const SourceMgr &SM, StringRef Buffer,
// If this match had "not strings", verify that they don't exist in the
// skipped region.
- if (CheckNot(SM, SkippedRegion, NotStrings, VariableTable, Req, Diags))
+ if (CheckNot(SM, SkippedRegion, NotStrings, Req, Diags))
return StringRef::npos;
}
@@ -1165,22 +2200,21 @@ bool FileCheckString::CheckSame(const SourceMgr &SM, StringRef Buffer) const {
bool FileCheckString::CheckNot(
const SourceMgr &SM, StringRef Buffer,
const std::vector<const FileCheckPattern *> &NotStrings,
- StringMap<StringRef> &VariableTable, const FileCheckRequest &Req,
- std::vector<FileCheckDiag> *Diags) const {
+ const FileCheckRequest &Req, std::vector<FileCheckDiag> *Diags) const {
for (const FileCheckPattern *Pat : NotStrings) {
assert((Pat->getCheckTy() == Check::CheckNot) && "Expect CHECK-NOT!");
size_t MatchLen = 0;
- size_t Pos = Pat->Match(Buffer, MatchLen, VariableTable);
+ size_t Pos = Pat->Match(Buffer, MatchLen, SM);
if (Pos == StringRef::npos) {
PrintNoMatch(false, SM, Prefix, Pat->getLoc(), *Pat, 1, Buffer,
- VariableTable, Req.VerboseVerbose, Diags);
+ Req.VerboseVerbose, Diags);
continue;
}
- PrintMatch(false, SM, Prefix, Pat->getLoc(), *Pat, 1, Buffer, VariableTable,
- Pos, MatchLen, Req, Diags);
+ PrintMatch(false, SM, Prefix, Pat->getLoc(), *Pat, 1, Buffer, Pos, MatchLen,
+ Req, Diags);
return true;
}
@@ -1192,7 +2226,6 @@ bool FileCheckString::CheckNot(
size_t
FileCheckString::CheckDag(const SourceMgr &SM, StringRef Buffer,
std::vector<const FileCheckPattern *> &NotStrings,
- StringMap<StringRef> &VariableTable,
const FileCheckRequest &Req,
std::vector<FileCheckDiag> *Diags) const {
if (DagNotStrings.empty())
@@ -1233,19 +2266,19 @@ FileCheckString::CheckDag(const SourceMgr &SM, StringRef Buffer,
// CHECK-DAG group.
for (auto MI = MatchRanges.begin(), ME = MatchRanges.end(); true; ++MI) {
StringRef MatchBuffer = Buffer.substr(MatchPos);
- size_t MatchPosBuf = Pat.Match(MatchBuffer, MatchLen, VariableTable);
+ size_t MatchPosBuf = Pat.Match(MatchBuffer, MatchLen, SM);
// With a group of CHECK-DAGs, a single mismatching means the match on
// that group of CHECK-DAGs fails immediately.
if (MatchPosBuf == StringRef::npos) {
PrintNoMatch(true, SM, Prefix, Pat.getLoc(), Pat, 1, MatchBuffer,
- VariableTable, Req.VerboseVerbose, Diags);
+ Req.VerboseVerbose, Diags);
return StringRef::npos;
}
// Re-calc it as the offset relative to the start of the original string.
MatchPos += MatchPosBuf;
if (Req.VerboseVerbose)
- PrintMatch(true, SM, Prefix, Pat.getLoc(), Pat, 1, Buffer,
- VariableTable, MatchPos, MatchLen, Req, Diags);
+ PrintMatch(true, SM, Prefix, Pat.getLoc(), Pat, 1, Buffer, MatchPos,
+ MatchLen, Req, Diags);
MatchRange M{MatchPos, MatchPos + MatchLen};
if (Req.AllowDeprecatedDagOverlap) {
// We don't need to track all matches in this mode, so we just maintain
@@ -1288,8 +2321,8 @@ FileCheckString::CheckDag(const SourceMgr &SM, StringRef Buffer,
MatchPos = MI->End;
}
if (!Req.VerboseVerbose)
- PrintMatch(true, SM, Prefix, Pat.getLoc(), Pat, 1, Buffer, VariableTable,
- MatchPos, MatchLen, Req, Diags);
+ PrintMatch(true, SM, Prefix, Pat.getLoc(), Pat, 1, Buffer, MatchPos,
+ MatchLen, Req, Diags);
// Handle the end of a CHECK-DAG group.
if (std::next(PatItr) == PatEnd ||
@@ -1300,7 +2333,7 @@ FileCheckString::CheckDag(const SourceMgr &SM, StringRef Buffer,
// region.
StringRef SkippedRegion =
Buffer.slice(StartPos, MatchRanges.begin()->Pos);
- if (CheckNot(SM, SkippedRegion, NotStrings, VariableTable, Req, Diags))
+ if (CheckNot(SM, SkippedRegion, NotStrings, Req, Diags))
return StringRef::npos;
// Clear "not strings".
NotStrings.clear();
@@ -1364,16 +2397,103 @@ Regex llvm::FileCheck::buildCheckPrefixRegex() {
return Regex(PrefixRegexStr);
}
-// Remove local variables from \p VariableTable. Global variables
-// (start with '$') are preserved.
-static void ClearLocalVars(StringMap<StringRef> &VariableTable) {
- SmallVector<StringRef, 16> LocalVars;
- for (const auto &Var : VariableTable)
+/// Define pattern and numeric variables from definitions given on the command
+/// line passed as a vector of VAR=VAL strings in \p CmdlineDefines. Report
+/// any error to \p SM.
+bool FileCheckPatternContext::DefineCmdlineVariables(
+ std::vector<std::string> &CmdlineDefines, SourceMgr &SM) {
+ static bool CmdlineVariablesDefined = false;
+
+ if (CmdlineVariablesDefined)
+ return true;
+ CmdlineVariablesDefined = true;
+
+ // Dummy pattern to call ParseNumericExpression
+ FileCheckPattern P(Check::CheckPlain, this);
+
+ bool ErrorFound = false;
+ for (const auto &CmdlineDef : CmdlineDefines) {
+ // Numeric variable definition. Same format as numeric expression with
+ // variable definition where ':' is replaced by '='
+ if (CmdlineDef[0] == '#') {
+ // Copy command-line definition text with leading '#' stripped and
+ // replace '=' by ':' to be able to reuse ParseNumericExpression.
+ // Create a buffer with fake command line content in order to display
+ // parsing diagnostic with location information and point to the
+ // command-line option with invalid syntax.
+ std::string Prefix = "-D";
+ std::string Suffix1 = " (parsed as: [[";
+ std::string Suffix2 = CmdlineDef;
+ size_t EqIdx = Suffix2.find('=');
+ Suffix2[EqIdx] = ':';
+ std::string Suffix3 = "]])";
+ std::string DiagCmdline =
+ Prefix + CmdlineDef + Suffix1 + Suffix2 + Suffix3;
+ std::unique_ptr<MemoryBuffer> CmdLine =
+ MemoryBuffer::getMemBufferCopy(DiagCmdline, "command line");
+ StringRef DiagCmdlineDefRef = CmdLine->getBuffer();
+ SM.AddNewSourceBuffer(std::move(CmdLine), SMLoc());
+
+ // Now parse to both check syntax is correct and create the necessary
+ // class instance.
+ size_t NumExprStartIdx =
+ Prefix.size() + CmdlineDef.size() + Suffix1.size() + 1;
+ StringRef CmdlineDefRef =
+ DiagCmdlineDefRef.substr(NumExprStartIdx).drop_back(Suffix3.size());
+ std::shared_ptr<FileCheckNumExprVar> NumVarDef;
+ std::shared_ptr<FileCheckNumExpr> NumExpr =
+ P.ParseNumericExpression(CmdlineDefRef, NumVarDef, SM);
+ if (NumVarDef == nullptr) {
+ ErrorFound = true;
+ continue;
+ }
+ NumExpr->SetValueFromEval();
+
+ // Record this variable definition.
+ GlobalNumericVariableTable[NumVarDef->GetName()] = NumVarDef;
+ // Pattern variable definition
+ } else {
+ std::string Prefix = "-D";
+ std::string DiagCmdline = Prefix + CmdlineDef;
+ std::unique_ptr<MemoryBuffer> CmdLine =
+ MemoryBuffer::getMemBufferCopy(DiagCmdline, "command line");
+ StringRef CmdlineDefRef = CmdLine->getBuffer().substr(Prefix.size());
+ SM.AddNewSourceBuffer(std::move(CmdLine), SMLoc());
+
+ bool IsPseudo;
+ unsigned TrailIdx;
+ std::pair<StringRef, StringRef> CmdlineNameVal = CmdlineDefRef.split('=');
+ StringRef Name = CmdlineNameVal.first;
+ if (P.ParseVariable(Name, IsPseudo, TrailIdx) || IsPseudo ||
+ TrailIdx != Name.size()) {
+ SM.PrintMessage(SMLoc::getFromPointer(CmdlineDefRef.data()),
+ SourceMgr::DK_Error,
+ "Invalid name for variable definition '" + Name + "'");
+ ErrorFound = true;
+ continue;
+ }
+ GlobalVariableTable.insert(CmdlineDefRef.split('='));
+ }
+ }
+
+ return ErrorFound;
+}
+
+/// Undefine local variables (variables whose name does not start with a '$'
+/// sign), ie remove them from GlobalVariableTable.
+void FileCheckPatternContext::ClearLocalVars(void) {
+ SmallVector<StringRef, 16> LocalPatternVars, LocalNumericVars;
+ for (const auto &Var : GlobalVariableTable)
+ if (Var.first()[0] != '$')
+ LocalPatternVars.push_back(Var.first());
+ for (const auto &Var : GlobalNumericVariableTable)
if (Var.first()[0] != '$')
- LocalVars.push_back(Var.first());
+ LocalNumericVars.push_back(Var.first());
- for (const auto &Var : LocalVars)
- VariableTable.erase(Var);
+ for (const auto &Var : LocalPatternVars)
+ GlobalVariableTable.erase(Var);
+ for (const auto &Var : LocalNumericVars)
+ GlobalNumericVariableTable.erase(Var);
}
/// Check the input to FileCheck provided in the \p Buffer against the \p
@@ -1385,12 +2505,8 @@ bool llvm::FileCheck::CheckInput(SourceMgr &SM, StringRef Buffer,
std::vector<FileCheckDiag> *Diags) {
bool ChecksFailed = false;
- /// VariableTable - This holds all the current filecheck variables.
- StringMap<StringRef> VariableTable;
-
- for (const auto& Def : Req.GlobalDefines)
- VariableTable.insert(StringRef(Def).split('='));
-
+ FileCheckPatternContext *Context =
+ CheckStrings.empty() ? nullptr : CheckStrings[0].Pat.GetContext();
unsigned i = 0, j = 0, e = CheckStrings.size();
while (true) {
StringRef CheckRegion;
@@ -1405,10 +2521,10 @@ bool llvm::FileCheck::CheckInput(SourceMgr &SM, StringRef Buffer,
// Scan to next CHECK-LABEL match, ignoring CHECK-NOT and CHECK-DAG
size_t MatchLabelLen = 0;
- size_t MatchLabelPos = CheckLabelStr.Check(
- SM, Buffer, true, MatchLabelLen, VariableTable, Req, Diags);
+ size_t MatchLabelPos =
+ CheckLabelStr.Check(SM, Buffer, true, MatchLabelLen, Req, Diags);
if (MatchLabelPos == StringRef::npos)
- // Immediately bail of CHECK-LABEL fails, nothing else we can do.
+ // Immediately bail if CHECK-LABEL fails, nothing else we can do.
return false;
CheckRegion = Buffer.substr(0, MatchLabelPos + MatchLabelLen);
@@ -1416,8 +2532,8 @@ bool llvm::FileCheck::CheckInput(SourceMgr &SM, StringRef Buffer,
++j;
}
- if (Req.EnableVarScope)
- ClearLocalVars(VariableTable);
+ if (Req.EnableVarScope && Context != nullptr)
+ Context->ClearLocalVars();
for (; i != j; ++i) {
const FileCheckString &CheckStr = CheckStrings[i];
@@ -1425,8 +2541,8 @@ bool llvm::FileCheck::CheckInput(SourceMgr &SM, StringRef Buffer,
// Check each string within the scanned region, including a second check
// of any final CHECK-LABEL (to verify CHECK-NOT and CHECK-DAG)
size_t MatchLen = 0;
- size_t MatchPos = CheckStr.Check(SM, CheckRegion, false, MatchLen,
- VariableTable, Req, Diags);
+ size_t MatchPos =
+ CheckStr.Check(SM, CheckRegion, false, MatchLen, Req, Diags);
if (MatchPos == StringRef::npos) {
ChecksFailed = true;
@@ -1440,6 +2556,7 @@ bool llvm::FileCheck::CheckInput(SourceMgr &SM, StringRef Buffer,
if (j == e)
break;
}
+ delete Context;
// Success if no checks failed.
return !ChecksFailed;
diff --git a/test/FileCheck/defines.txt b/test/FileCheck/defines.txt
index 5a2ca83457b..5fd0daba7f8 100644
--- a/test/FileCheck/defines.txt
+++ b/test/FileCheck/defines.txt
@@ -1,21 +1,57 @@
; RUN: FileCheck -DVALUE=10 -input-file %s %s
; RUN: not FileCheck -DVALUE=20 -input-file %s %s 2>&1 | FileCheck %s -check-prefix ERRMSG
-;
; RUN: not FileCheck -DVALUE=10 -check-prefix NOT -input-file %s %s 2>&1 | FileCheck %s -check-prefix NOT-ERRMSG
; RUN: FileCheck -DVALUE=20 -check-prefix NOT -input-file %s %s
; RUN: not FileCheck -DVALUE10 -input-file %s %s 2>&1 | FileCheck %s -check-prefix ERRCLI
+; RUN: not FileCheck -D10VALUE=10 -input-file %s %s 2>&1 | FileCheck %s -check-prefix ERRCLIFMT
+; RUN: not FileCheck -D@VALUE=10 -input-file %s %s 2>&1 | FileCheck %s -check-prefix ERRCLIPSEUDO
+; RUN: not FileCheck -D'VALUE + 2=10' -input-file %s %s 2>&1 | FileCheck %s -check-prefix ERRCLITRAIL
+; RUN: FileCheck -D#%X,NUMVAL1=8 -D#NUMVAL2=NUMVAL1+4 -check-prefix CHECKNUM -input-file %s %s
+; RUN: not FileCheck -D#%X,NUMVAL1=8 -D#NUMVAL2=NUMVAL1+6 -check-prefix CHECKNUM -input-file %s %s 2>&1 | FileCheck %s -check-prefix NUMERRMSG
+; RUN: not FileCheck -D#%X,NUMVAL1=8 -D#NUMVAL2=NUMVAL1+4 -check-prefix NUMNOT -input-file %s %s 2>&1 | FileCheck %s -check-prefix NOT-NUMERRMSG
+; RUN: FileCheck -D#%X,NUMVAL1=8 -D#NUMVAL2=NUMVAL1+6 -check-prefixes NUMNOT -input-file %s %s
+; RUN: not FileCheck -D#,VALUE=10 -input-file %s %s 2>&1 | FileCheck %s -check-prefix NUMERRCLI
Value = 10
; CHECK: Value = [[VALUE]]
; NOT-NOT: Value = [[VALUE]]
-; ERRMSG: defines.txt:9:10: error: CHECK: expected string not found in input
+; ERRMSG: defines.txt:[[#@LINE-3]]:10: error: CHECK: expected string not found in input
; ERRMSG: defines.txt:1:1: note: scanning from here
; ERRMSG: defines.txt:1:1: note: with variable "VALUE" equal to "20"
-; ERRMSG: defines.txt:8:1: note: possible intended match here
+; ERRMSG: defines.txt:[[#@LINE-7]]:1: note: possible intended match here
-; NOT-ERRMSG: defines.txt:10:12: error: {{NOT}}-NOT: excluded string found in input
-; NOT-ERRMSG: defines.txt:8:1: note: found here
-; NOT-ERRMSG: defines.txt:8:1: note: with variable "VALUE" equal to "10"
+; NOT-ERRMSG: defines.txt:[[#@LINE-7]]:12: error: {{NOT}}-NOT: excluded string found in input
+; NOT-ERRMSG: defines.txt:[[#VALLINE:@LINE-10]]:1: note: found here
+; NOT-ERRMSG: defines.txt:[[#VALLINE]]:1: note: with variable "VALUE" equal to "10"
; ERRCLI: Command-line definition 'VALUE10' missing equal sign
+
+; ERRCLIFMT: command line:1:3: error: Invalid name for variable definition '10VALUE'
+; ERRCLIFMT-NEXT: -D10VALUE=10
+; ERRCLIFMT-NEXT: {{^ \^$}}
+
+; ERRCLIPSEUDO: command line:1:3: error: Invalid name for variable definition '@VALUE'
+; ERRCLIPSEUDO-NEXT: -D@VALUE=10
+; ERRCLIPSEUDO-NEXT: {{^ \^$}}
+
+; ERRCLITRAIL: command line:1:3: error: Invalid name for variable definition 'VALUE + 2'
+; ERRCLITRAIL-NEXT: -DVALUE + 2=10
+; ERRCLITRAIL-NEXT: {{^ \^$}}
+
+Numeric value #2 = C
+; CHECKNUM: Numeric value #2 = [[#NUMVAL2]]
+; NUMNOT-NOT: Numeric value #2 = [[#NUMVAL2]]
+
+; NUMERRMSG: defines.txt:[[#@LINE-3]]:13: error: CHECKNUM: expected string not found in input
+; NUMERRMSG: defines.txt:1:1: note: scanning from here
+; NUMERRMSG: defines.txt:1:1: note: with numeric expression "NUMVAL2" equal to "E"
+; NUMERRMSG: defines.txt:[[#@LINE-7]]:1: note: possible intended match here
+
+; NOT-NUMERRMSG: defines.txt:[[#@LINE-7]]:15: error: {{NUMNOT}}-NOT: excluded string found in input
+; NOT-NUMERRMSG: defines.txt:[[#NUMVALLINE:@LINE-10]]:1: note: found here
+; NOT-NUMERRMSG: defines.txt:[[#NUMVALLINE]]:1: note: with numeric expression "NUMVAL2" equal to "C"
+
+; NUMERRCLI: command line:1:29: error: Invalid matching format specification in numeric expression
+; NUMERRCLI-NEXT: -D#,VALUE=10 (parsed as: {{\[\[#,VALUE:10\]\]}})
+; NUMERRCLI-NEXT: {{^ \^$}}
diff --git a/test/FileCheck/numeric-expression.txt b/test/FileCheck/numeric-expression.txt
new file mode 100644
index 00000000000..cfa72ffec52
--- /dev/null
+++ b/test/FileCheck/numeric-expression.txt
@@ -0,0 +1,346 @@
+; RUN: FileCheck -input-file %s %s
+; RUN: not FileCheck -check-prefixes ERR,ERR1 -input-file %s %s 2>&1 | FileCheck -check-prefix ERRMSG1 %s
+; RUN: not FileCheck -check-prefixes ERR,ERR2 -input-file %s %s 2>&1 | FileCheck -check-prefix ERRMSG2 %s
+; RUN: not FileCheck -check-prefixes ERR,ERR3 -input-file %s %s 2>&1 | FileCheck -check-prefix ERRMSG3 %s
+; RUN: not FileCheck -check-prefixes ERR,ERR4 -input-file %s %s 2>&1 | FileCheck -check-prefix ERRMSG4 %s
+; RUN: not FileCheck -check-prefix ERR5 -input-file %s %s 2>&1 | FileCheck -check-prefix ERRMSG5 %s
+; RUN: not FileCheck -check-prefixes CHECK,ERR6 -input-file %s %s 2>&1 | FileCheck -check-prefix ERRMSG6 %s
+; RUN: not FileCheck -check-prefix ERR7 -input-file %s %s 2>&1 | FileCheck -check-prefix ERRMSG7 %s
+; RUN: not FileCheck -check-prefix ERR8 -input-file %s %s 2>&1 | FileCheck -check-prefix ERRMSG8 %s
+
+; We ensure we attemt to match all lines with digits by wrapping them between
+; START and END and using CHECK-NEXT to ensure each line is covered.
+
+
+; Numeric variable definition with default matching format
+DEF DEFAULT FMT
+11
+; CHECK-LABEL: DEF DEFAULT FMT
+; CHECK-NEXT: [[#VAR1:]]
+
+
+; Name conflict between Numeric variable definition and pattern variable definition
+PATVAR NUMVAR CONFLICT
+11
+foo1bar
+foo2bar
+22
+; ERR-LABEL: PATVAR NUMVAR CONFLICT
+; ERR1-NEXT: [[#PATVAR1:]]
+; ERR1-NEXT: [[PATVAR1:foo1.*]]
+; ERR2: [[PATVAR2:foo2.*]]
+; ERR2-NEXT: [[#PATVAR2:]]
+
+; ERRMSG1: numeric-expression.txt:[[#@LINE-4]]:16: error: Numeric variable with same name 'PATVAR1' exists
+; ERRMSG1-NEXT: ; {{E}}RR1-NEXT: {{\[\[PATVAR1:foo1\.\*\]\]}}
+; ERRMSG2: numeric-expression.txt:[[#@LINE-4]]:17: error: Pattern variable with same name 'PATVAR2' exists
+; ERRMSG2-NEXT: ; {{E}}RR2-NEXT: {{\[\[#PATVAR2:\]\]}}
+
+
+;; Numeric variable definition with default matching format in alternate spacing
+DEF DEFAULT FMT ALT SPC
+11
+11
+11
+; CHECK-LABEL: DEF DEFAULT FMT ALT SPC
+; CHECK-NEXT: [[# VAR1a:]]
+; CHECK-NEXT: [[# VAR1b :]]
+; CHECK-NEXT: [[# VAR1c : ]]
+
+; Numeric variable definition with explicit matching format
+DEF FMT
+-30
+c
+D
+; CHECK-LABEL: DEF FMT
+; CHECK-NEXT: [[#%d,VAR2:]]
+; CHECK-NEXT: [[#%x,VAR3:]]
+; CHECK-NEXT: [[#%X,VAR4:]]
+
+
+; Numeric variable definition with unsupported matching format
+DEF INVALID FMT
+INVVAR1=a
+INVVAR2=11
+; ERR-LABEL: DEF INVALID FMT
+; ERR3-NEXT: INVVAR1=[[#%c,INVVAR1:]]
+; ERR4-NEXT: INVVAR2=[[#%hhd,INVVAR2:]]
+; ERRMSG3: numeric-expression.txt:[[#@LINE-2]]:26: error: Invalid format specifier in numeric expression
+; ERRMSG3-NEXT: ; {{E}}RR3-NEXT: INVVAR1={{\[\[#%c,INVVAR1:\]\]}}
+; ERRMSG3-NEXT: {{^ \^$}}
+; ERRMSG4: numeric-expression.txt:[[#@LINE-4]]:26: error: Invalid format specifier in numeric expression
+; ERRMSG4-NEXT: ; {{E}}RR4-NEXT: INVVAR2={{\[\[#%hhd,INVVAR2:\]\]}}
+; ERRMSG4-NEXT: {{^ \^$}}
+
+
+; Numeric variable definition with explicit matching format in alternate spacing
+DEF FMT ALT SPC
+-30
+-30
+-30
+-30
+-30
+; CHECK-LABEL: DEF FMT ALT SPC
+; CHECK-NEXT: [[#%d, VAR2a:]]
+; CHECK-NEXT: [[# %d, VAR2b:]]
+; CHECK-NEXT: [[# %d , VAR2c:]]
+; CHECK-NEXT: [[# %d , VAR2d :]]
+; CHECK-NEXT: [[# %d , VAR2e : ]]
+
+; Numeric expressions in explicit matching format and default matching rule using
+; variables defined on other lines
+USE DEF FMT IMPL MATCH
+11
+12
+10
+-30
+-29
+-31
+c
+d
+b
+1a
+1a
+D
+E
+C
+1B
+1B
+; CHECK-LABEL: USE DEF FMT IMPL MATCH
+; CHECK-NEXT: [[#%u,VAR1]]
+; CHECK-NEXT: [[#%u,VAR1+1]]
+; CHECK-NEXT: [[#%u,VAR1-1]]
+; CHECK-NEXT: [[#%d,VAR2]]
+; CHECK-NEXT: [[#%d,VAR2+1]]
+; CHECK-NEXT: [[#%d,VAR2-1]]
+; CHECK-NEXT: [[#%x,VAR3]]
+; CHECK-NEXT: [[#%x,VAR3+1]]
+; CHECK-NEXT: [[#%x,VAR3-1]]
+; CHECK-NEXT: [[#%x,VAR3+0xe]]
+; CHECK-NEXT: [[#%x,VAR3+0xE]]
+; CHECK-NEXT: [[#%X,VAR4]]
+; CHECK-NEXT: [[#%X,VAR4+1]]
+; CHECK-NEXT: [[#%X,VAR4-1]]
+; CHECK-NEXT: [[#%X,VAR4+0xe]]
+; CHECK-NEXT: [[#%X,VAR4+0xE]]
+
+
+; Numeric expression using undefined variable
+UNDEF VAR USE
+UNDEFVAR: 11
+; ERR5-LABEL: UNDEF VAR USE
+; ERR5-NEXT: UNDEFVAR: [[#UNDEFVAR]]
+; ERRMSG5: numeric-expression.txt:[[#@LINE-1]]:27: error: Using undefined numeric variable 'UNDEFVAR'
+; ERRMSG5-NEXT: ; {{E}}RR5-NEXT: UNDEFVAR: {{\[\[#UNDEFVAR\]\]}}
+; ERRMSG5-NEXT: {{^ \^$}}
+
+
+; Numeric expression with unsupported operator
+INVALID OPERATOR
+VAR1*2: 22
+; ERR6-LABEL: INVALID OPERATOR
+; ERR6-NEXT: VAR1*2: [[#VAR1*2]]
+; ERRMSG6: numeric-expression.txt:[[#@LINE-1]]:29: error: Unsupported numeric operation '*'
+; ERRMSG6-NEXT: ; {{E}}RR6-NEXT: VAR1*2: {{\[\[#VAR1\*2\]\]}}
+; ERRMSG6-NEXT: {{^ \^$}}
+
+
+; Numeric expression with overflow
+OVERFLOW
+BIGVAR=10000000000000000
+; ERR7-LABEL: OVERFLOW
+; ERR7-NEXT: BIGVAR: [[#BIGVAR:0x8000000000000000+0x8000000000000000]]
+; ERRMSG7: numeric-expression.txt:[[#@LINE-1]]:25: error: Failed to evaluate numeric expression known at parse time
+; ERRMSG7-NEXT: {{E}}RR7-NEXT: BIGVAR: {{\[\[#BIGVAR:0x8000000000000000\+0x8000000000000000\]\]}}
+; ERRMSG7-NEXT: {{^ \^$}}
+
+
+; Numeric expression with underflow
+UNDERFLOW
+TINYVAR=-10000000000000000
+; ERR8-LABEL: UNDERFLOW
+; ERR8-NEXT: TINYVAR: [[#%d,TINYVAR:-0x8000000000000000-0x8000000000000000]]
+; ERRMSG8: numeric-expression.txt:[[#@LINE-1]]:26: error: Failed to evaluate numeric expression known at parse time
+; ERRMSG8-NEXT: {{E}}RR8-NEXT: TINYVAR: {{\[\[#%d,TINYVAR:-0x8000000000000000-0x8000000000000000\]\]}}
+; ERRMSG8-NEXT: {{^ \^$}}
+
+
+; Numeric expressions in explicit matching format and default matching rule using
+; variables defined on other lines in alternate spacing
+USE EXPL FMT IMPL MATCH ALT SPC
+11
+11
+11
+12
+12
+12
+12
+12
+12
+10
+10
+10
+10
+10
+10
+; CHECK-LABEL: USE EXPL FMT IMPL MATCH ALT SPC
+; CHECK-NEXT: [[#%u, VAR1]]
+; CHECK-NEXT: [[# %u, VAR1]]
+; CHECK-NEXT: [[# %u, VAR1 ]]
+; CHECK-NEXT: [[#%u, VAR1+1]]
+; CHECK-NEXT: [[# %u, VAR1+1]]
+; CHECK-NEXT: [[# %u , VAR1+1]]
+; CHECK-NEXT: [[# %u , VAR1 +1]]
+; CHECK-NEXT: [[# %u , VAR1 + 1]]
+; CHECK-NEXT: [[# %u , VAR1 + 1 ]]
+; CHECK-NEXT: [[#%u, VAR1-1]]
+; CHECK-NEXT: [[# %u, VAR1-1]]
+; CHECK-NEXT: [[# %u , VAR1-1]]
+; CHECK-NEXT: [[# %u , VAR1 -1]]
+; CHECK-NEXT: [[# %u , VAR1 - 1]]
+; CHECK-NEXT: [[# %u , VAR1 - 1 ]]
+
+
+; Numeric expressions in implicit matching format and default matching rule using
+; variables defined on other lines
+USE IMPL FMT IMPL MATCH
+11
+12
+10
+-30
+-29
+-31
+c
+d
+b
+1a
+1a
+D
+E
+C
+1B
+1B
+; CHECK-LABEL: USE IMPL FMT IMPL MATCH
+; CHECK-NEXT: [[#VAR1]]
+; CHECK-NEXT: [[#VAR1+1]]
+; CHECK-NEXT: [[#VAR1-1]]
+; CHECK-NEXT: [[#VAR2]]
+; CHECK-NEXT: [[#VAR2+1]]
+; CHECK-NEXT: [[#VAR2-1]]
+; CHECK-NEXT: [[#VAR3]]
+; CHECK-NEXT: [[#VAR3+1]]
+; CHECK-NEXT: [[#VAR3-1]]
+; CHECK-NEXT: [[#VAR3+0xe]]
+; CHECK-NEXT: [[#VAR3+0xE]]
+; CHECK-NEXT: [[#VAR4]]
+; CHECK-NEXT: [[#VAR4+1]]
+; CHECK-NEXT: [[#VAR4-1]]
+; CHECK-NEXT: [[#VAR4+0xe]]
+; CHECK-NEXT: [[#VAR4+0xE]]
+
+
+; Numeric expressions in default matching format and default matching rule
+; using variables defined on other lines and an immediate interpreted as an
+; unsigned value
+; Note: 9223372036854775819 = 0x8000000000000000 + 11
+USE IMPL FMT IMPL MATCH UNSIGNED IMM
+9223372036854775819
+; CHECK-LABEL: USE IMPL FMT IMPL MATCH UNSIGNED IMM
+; CHECK-NEXT: [[#VAR1+0x8000000000000000]]
+
+
+; Numeric expressions in default matching format and explicit matching rule using
+; variables defined on other lines
+USE DEF FMT EXPL MATCH
+11
+11
+11
+; CHECK-LABEL: USE DEF FMT EXPL MATCH
+; CHECK-NEXT: [[#==VAR1]]
+; CHECK-NEXT: [[# ==VAR1]]
+; CHECK-NEXT: [[# == VAR1]]
+
+
+; Numeric expressions with conversion matching format and implicit matching rule
+; using variables defined on other lines
+USE CONV FMT IMPL MATCH
+b
+B
+12
+12
+13
+13
+; CHECK-LABEL: USE CONV FMT IMPL MATCH
+; CHECK-NEXT: [[# %x, VAR1]]
+; CHECK-NEXT: [[# %X, VAR1]]
+; CHECK-NEXT: [[# %u, VAR3]]
+; CHECK-NEXT: [[# %d, VAR3]]
+; CHECK-NEXT: [[# %u, VAR4]]
+; CHECK-NEXT: [[# %d, VAR4]]
+
+
+; Numeric expression with explicit and default matching format and implicit
+; matching rule using variables defined on same line
+USE SAME EXPL FMT IMPL MATCH
+11 10
+-11 C
+b c
+b 12 c
+C B
+B 10
+; CHECK-LABEL: USE SAME EXPL FMT IMPL MATCH
+; CHECK-NEXT: [[#VAR5:]] [[#%u,VAR5-1]]
+; CHECK-NEXT: [[#%d,VAR6:]] [[#%X,VAR6+23]]
+; CHECK-NEXT: [[#%x,VAR7:]] [[#%x,VAR7+1]]
+; CHECK-NEXT: [[#%x,VAR8:]] [[#%u,VAR8+1]] [[#%x,VAR8+1]]
+; CHECK-NEXT: [[#%X,VAR9:]] [[#%X,VAR9-1]]
+; CHECK-NEXT: [[#%X,VAR10:]] [[#%d,VAR10-1]]
+
+
+; Numeric expression with implicit matching format and implicit matching rule
+; using variables defined on same line
+USE SAME IMPL FMT IMPL MATCH
+11 10
+-11 -4
+b c
+b 12 c
+C B
+; CHECK-LABEL: USE SAME IMPL FMT IMPL MATCH
+; CHECK-NEXT: [[#VAR5:]] [[#VAR5-1]]
+; CHECK-NEXT: [[#%d,VAR6:]] [[#VAR6+7]]
+; CHECK-NEXT: [[#%x,VAR7:]] [[#VAR7+1]]
+; CHECK-NEXT: [[#%x,VAR8:]] [[#%u,VAR8+1]] [[#VAR8+1]]
+; CHECK-NEXT: [[#%X,VAR9:]] [[#VAR9-1]]
+
+
+; Numeric expression with implicit matching format and implicit matching rule
+; using variables defined on same line satisfying constraint is not the first
+; line encountered.
+REDEF SAME IMPL FMT IMPL MATCH NOT FIRST
+1 3
+1 2
+; CHECK-LABEL: REDEF SAME IMPL FMT IMPL MATCH NOT FIRST
+; CHECK: [[#N:]] [[#N+1]]
+
+
+; Numeric expression with explicit matching format and implicit matching rule
+; using variables redefined on same line
+REDEF EXPL FMT IMPL MATCH
+21
+22 -31 -32
+AC/DC dc db
+; CHECK-LABEL: REDEF EXPL FMT IMPL MATCH
+; CHECK-NEXT: [[#VAR11:]]
+; CHECK-NEXT: [[#%u,VAR11+1]] [[#%d,VAR11:]] [[#%d,VAR11-1]]
+; CHECK-NEXT: [[#%X,VAR12:]]/[[#%X,VAR12+48]] [[#%x,VAR12:]] [[#%x,VAR12-1]]
+
+; Numeric expression with implicit matching format and implicit matching rule
+; using variables redefined on same line
+REDEF IMPL FMT IMPL MATCH
+21
+22 -31 -32
+AC/DC dc db
+; CHECK-LABEL: REDEF IMPL FMT IMPL MATCH
+; CHECK-NEXT: [[#VAR11:]]
+; CHECK-NEXT: [[#VAR11+1]] [[#%d,VAR11:]] [[#VAR11-1]]
+; CHECK-NEXT: [[#%X,VAR12:]]/[[#VAR12+0x30]] [[#%x,VAR12:]] [[#VAR12-1]]
diff --git a/test/FileCheck/regex-scope.txt b/test/FileCheck/regex-scope.txt
deleted file mode 100644
index 989f422c6bc..00000000000
--- a/test/FileCheck/regex-scope.txt
+++ /dev/null
@@ -1,23 +0,0 @@
-// RUN: FileCheck -input-file %s %s
-// RUN: FileCheck -check-prefixes CHECK,GLOBAL -input-file %s %s
-// RUN: FileCheck -check-prefixes CHECK,LOCAL -input-file %s %s
-// RUN: FileCheck -check-prefixes CHECK,GLOBAL --enable-var-scope -input-file %s %s
-// RUN: not FileCheck -check-prefixes CHECK,LOCAL --enable-var-scope -input-file %s %s
-
-local
-global
-; CHECK: [[LOCAL:loc.*]]
-; CHECK: [[$GLOBAL:glo.*]]
-
-local2
-global2
-; CHECK: [[LOCAL]]2
-; CHECK: [[$GLOBAL]]2
-
-barrier:
-; CHECK-LABEL: barrier
-
-local3
-global3
-; LOCAL: [[LOCAL]]3
-; GLOBAL: [[$GLOBAL]]3
diff --git a/test/FileCheck/var-scope.txt b/test/FileCheck/var-scope.txt
new file mode 100644
index 00000000000..c95395c521b
--- /dev/null
+++ b/test/FileCheck/var-scope.txt
@@ -0,0 +1,26 @@
+// RUN: FileCheck -input-file %s %s
+// RUN: FileCheck -check-prefixes CHECK,GLOBAL -input-file %s %s
+// RUN: FileCheck -check-prefixes CHECK,LOCAL,LOCAL3 -input-file %s %s
+// RUN: FileCheck -check-prefixes CHECK,GLOBAL --enable-var-scope -input-file %s %s
+// RUN: not FileCheck -check-prefixes CHECK,LOCAL,LOCAL1 --enable-var-scope -input-file %s %s
+// RUN: not FileCheck -check-prefixes CHECK,LOCAL,LOCAL2 --enable-var-scope -input-file %s %s
+
+local1
+global1
+; CHECK: [[LOCAL:loc[^[:digit:]]*]][[#LOCNUM:]]
+; CHECK: [[$GLOBAL:glo[^[:digit:]]*]][[#GLOBNUM:]]
+
+local2
+global2
+; CHECK: [[LOCAL]][[#LOCNUM+1]]
+; CHECK: [[$GLOBAL]][[#GLOBNUM+1]]
+
+barrier:
+; CHECK-LABEL: barrier
+
+local3
+global3
+; LOCAL1: [[LOCAL]]3
+; LOCAL2: local[[#LOCNUM+2]]
+; LOCAL2: [[LOCAL]][[#LOCNUM+2]]
+; GLOBAL: [[$GLOBAL]][[#GLOBNUM+2]]
diff --git a/test/FileCheck/verbose.txt b/test/FileCheck/verbose.txt
index e6eec5e5462..308488391ff 100644
--- a/test/FileCheck/verbose.txt
+++ b/test/FileCheck/verbose.txt
@@ -29,6 +29,39 @@ VV-NEXT: verbose.txt:[[@LINE-22]]:1: note: scanning from here
VV-NEXT: {{^}}bar{{$}}
VV-NEXT: {{^}}^{{$}}
+VAR1=42
+VAR1:42 VAR1=12 VAR1:12
+CHECK: VAR1=[[#VAR1:]]
+CHECK-NOT: [[#VAR1 + 1]]
+CHECK-NEXT: VAR1:[[#VAR1]] VAR1=[[#VAR1:]] VAR1:[[#VAR1]]
+
+V: verbose.txt:[[#@LINE-4]]:8: remark: {{C}}HECK: expected string found in input
+V-NEXT: {{C}}HECK: {{VAR1=[[][[]#VAR1:[]][]]$}}
+V-NEXT: {{^ \^$}}
+V-NEXT: verbose.txt:[[#@LINE-9]]:1: note: found here
+V-NEXT: {{^}}VAR1=42{{$}}
+V-NEXT: {{^}}^~~~~~~{{$}}
+
+V-NEXT: verbose.txt:[[#@LINE-9]]:13: remark: {{C}}HECK-NEXT: expected string found in input
+V-NEXT: {{C}}HECK-NEXT: {{VAR1:[[][[]#VAR1[]][]] VAR1=[[][[]#VAR1:[]][]] VAR1:[[][[]#VAR1[]][]]$}}
+V-NEXT: {{^ \^$}}
+V-NEXT: verbose.txt:[[#VARLIN1:@LINE-15]]:1: note: found here
+V-NEXT: {{^}}VAR1:42 VAR1=12 VAR1:12{{$}}
+V-NEXT: {{^}}^~~~~~~~~~~~~~~~~~~~~~~{{$}}
+VV-NEXT: verbose.txt:[[#VARLIN1]]:1: note: with numeric expression "VAR1" equal to "42"
+VV-NEXT: {{^}}VAR1:42 VAR1=12 VAR1:12{{$}}
+VV-NEXT: {{^}}^~~~~~~~~~~~~~~~~~~~~~~{{$}}
+VV-NEXT: verbose.txt:[[#VARLIN1]]:1: note: with numeric expression "VAR1" equal to "12"
+VV-NEXT: {{^}}VAR1:42 VAR1=12 VAR1:12{{$}}
+VV-NEXT: {{^}}^~~~~~~~~~~~~~~~~~~~~~~{{$}}
+
+VV-NEXT: verbose.txt:[[#@LINE-23]]:12: remark: {{C}}HECK-NOT: excluded string not found in input
+VV-NEXT: {{C}}HECK-NOT: {{[[][[]#VAR1 [+] 1[]][]]$}}
+VV-NEXT: {{^ \^$}}
+VV-NEXT: verbose.txt:[[#@LINE-28]]:1: note: scanning from here
+VV-NEXT: {{^}}VAR1:42 VAR1=12 VAR1:12{{$}}
+VV-NEXT: {{^}}^{{$}}
+
before empty
after empty