Tiny JavaScript tokenizer.
Package: js-tokens
Created by: lydell
Last modified: Thu, 08 Feb 2024 22:59:07 GMT
Version: 9.0.0
License: MIT
Downloads: 186,254,304
Repository: https://github.com/lydell/js-tokens
Install
npm install js-tokens
yarn add js-tokens
js-tokens
The tiny, regex powered, lenient, almost spec-compliant JavaScript tokenizer that never fails.
const jsTokens = require("js-tokens");
const jsString = 'JSON.stringify({k:3.14**2}, null /*replacer*/, "\\t")';
Array.from(jsTokens(jsString), (token) => token.value).join("|");
// JSON|.|stringify|(|{|k|:|3.14|**|2|}|,| |null| |/*replacer*/|,| |"\t"|)
