Lexical structure

Names, comments, number, string and character literals, newlines, line splices and digraphs: what the characters of a source file mean before any grammar applies.

The literals follow C++, and where PaskalCPP differs this page says so; a Pascal developer will notice that PaskalCPP is case-sensitive and that its comments are C++'s. Where a literal's value goes once it is read (its type, constant folding) is on Expressions.

module exe lexical_literals;

var
  BEGIN: int32 = 3;

begin
  println("suffixes {} {} {} {} {} {}", 7lu, 7LLU, 7uz, 7Z, 7U, 0B11);
  println("floats {} {} {}", 1.5F, 1.5L, .5f);
  println("hex floats {} {} {}", 0x1p-2, 0X1P4, 0x.8p1);
  println("delimited {} {} {}", "\u{e9}", "\x{41}", "\o{101}");
  println("named {}", "\N{LATIN SMALL LETTER E WITH ACUTE}");
  println("astral {}", "\U0001F600");
  println("prefixed len {} {} {} {}", len(u8"abc"), len(u"abc"), len(U"abc"), len(L"abc"));
  println("wide raw {}", LR"(c\d)");
  println("delimiter {}", R"abcdefghijklmnop(x)abcdefghijklmnop");
  /**/ println("name {}", BEGIN);
end.

suffixes 7 7 7 7 7 3
floats 1.5 1.5 0.5
hex floats 0.25 16 1
delimited é A A
named é
astral 😀
prefixed len 3 3 3 3
wide raw c\d
delimiter x
name 3
[Heap] Allocs: 0, Frees: 0, Leaked: 0

Names

A name is a letter (A..Z, a..z) or _, followed by letters, digits and _: _hidden and a_1_b are names.

PaskalCPP is case-sensitive: LValue, lvalue and LVALUE are three different names. Keywords are lowercase, so Begin and BEGIN are ordinary names; the testAssert* asserts keep their mixed case.

• A letter outside ASCII, outside a string or a comment, is LX0004 (Invalid character in source: é for café), as is a character no token starts with, such as a backtick.
• A byte that is not UTF-8 is LX0011.
• An @ with no directive name after it is LX0009.
• A source file may begin with a UTF-8 byte order mark.

Comments

// runs to the end of the line and /* ... */ to the first */. Two slashes inside a string are text, and /**/ is an empty comment.

Block comments do not nest: a /* inside one is text and the warning LX0203, and the first */ closes the comment. A block comment that is never closed is LX0003.

Integer literals

Integers are written as C++ writes them: decimal, octal (a leading 0: 010 is 8), hex (0x / 0X) and binary (0b / 0B). A ' separates digits: 1'000'000, 0xFF'FF.

• An octal literal gets the hint LX0101 (Leading zero: 010 is an octal literal); an 8 or 9 in one is LX0007. A digit the base does not have (0b12) is LX0005.
• The suffixes are C++'s: u / U, l / L, ll / LL, z / Z, and u with any of the others in either order (7ULL, 7lu, 7LLU, 7uz).
• A literal must fit a C++ type that holds its value (C++23). A decimal literal without u goes up to the int64 maximum; above it is LX0006, and the message names the fix: write 9223372036854775808u. A hex, octal or binary literal, or one with u, goes up to the uint64 maximum; above that every literal is LX0006.
• Any other suffix touching the literal is a C++ user-defined literal: 3_k calls the operator""_k that mixed C++ declares, and clang judges it.

Float literals

1.5, 1., .5, an exponent (1e3, 2.5e-2) and the ' separator (1'000.5) are floats. 1..5 is not a float: it is 1, .., 5, a range.

• A hex float needs its p exponent: 0x1.8p3 is 12, 0x1p-2 is 0.25; 0x1.8 with no p is LX0005.
• An exponent with no digits (1.5e) is LX0005; a value no float64 holds (1e400) is LX0006.
• The suffixes are f / F (a float32: 2.5f), l / L (long double) and f16 / F16 (a C++ _Float16). These are exactly the ones the bundled clang takes, checked on every supported target.
• C++23's other float suffixes, f32, f64, f128 (and their capitals) and bf16 / BF16, are refused by the bundled clang, so they are LX0005 at the literal.

Strings and characters

Adjacent strings join into one: "joined " "in" " one".

The escapes are C++'s, decoded as clang decodes them: \", \\, \n, hex \x41, octal \101, \u00e9, \U0001F600, the delimited forms \u{e9}, \x{41}, \o{101}, and named characters such as \N{LATIN SMALL LETTER E WITH ACUTE} (Unicode 15.1 names). An escape is one character: len("a\nb") is 3.

• An octal escape above 255 ("\400") is PR0009.
• An escape C++ does not know ("\q") is the warning PR0021, and the text keeps the q: "a\qb" prints aqb. In a char ('\q') it is the warning PR0064 and the character is the q.
• A raw string R"(...)" keeps its text as written (\n stays two characters) and may run over several lines. A delimiter (R"tag(has )" inside)tag") lets the text hold )"; it is at most 16 characters, with no space, parenthesis, backslash or control character, else LX0010.
• A string with no closing quote is LX0001; a raw string with no closing )" is LX0002.

A prefix touches the quote. L"wide" is a wstring, which prints as its text, and a raw string takes one too (LR"(c\d)"). u8"...", u"..." and U"..." are C++ strings: len counts them, but printing one is SM0425, since the C/C++ type has no formatter.

'c' is a char and L'w' a wchar; an empty '' is PR0054. A char literal of several characters is the warning PR0064 and is the int its bytes make ('ab' is 24930); L'ab' is PR0054.

Newlines and line splices

A newline is LF, CR LF or a lone CR, as in clang.

A backslash right before a newline is a line splice, removed before tokens are read: it joins a name (LVa\ and lue on the next line make LValue), a string, or a // comment with the next line. Lines and columns in diagnostics stay the physical ones.

• A space between the backslash and the newline is the warning LX0201, and it is still a splice, in code and in a // comment alike.
• A cppstart block whose last line ends in a splice is LX0014: the splice would join the next line the compiler writes.

Digraphs

C++'s digraphs work everywhere, PaskalCPP code included: <: :> are [ ] (LA<:1:> is LA[1]), <% %> are { }, and %: and %:%: are # and ##.

A %: first on its line starts a preprocessor line; anywhere else it is LX0008. <:: is < then ::, so std::vector<::std::int32_t> is a vector of int32.