Move rescript.json reading out of bsc into rewatch (#8365)

* Add -bs-project-root flag so bsc no longer walks for rescript.json

The compiler's Ext_path.package_dir lazily walked the filesystem from
cwd until it found rescript.json, solely to determine the JS output
root in lam_compile_main. That walk is unnecessary when rewatch is
driving the compile — rewatch already knows the package root.

- compiler: add custom_package_dir ref in Ext_path; convert
  package_dir to a unit -> string that prefers the ref and falls back
  to the existing walk. Add -bs-project-root CLI flag on bsc.
- rewatch: emit -bs-project-root <package-root> from compiler_args
  for every bsc invocation.

No behavioral change for users; this is the first step toward removing
direct rescript.json reads from bsc entirely.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Move gentype config from rescript.json to CLI flags

The gentype backend was the last site in bsc that parsed rescript.json
directly (via GenTypeConfig.read_config walking up from cwd). Rewatch
already parses that file, so it can hand the needed fields to bsc as
CLI flags, eliminating the filesystem walk and the duplicated JSON
reading.

Compiler side:
- GenTypeConfig.ml: drop the JSON reader. Replace with module-level
  refs populated by bsc CLI flags, plus `build_config ~namespace` that
  assembles a Config.t from those refs.
- New bsc flags: -bs-gentype-module, -bs-gentype-module-resolution,
  -bs-gentype-export-interfaces, -bs-gentype-generated-extension,
  -bs-gentype-suffix, -bs-gentype-shim, -bs-gentype-debug,
  -bs-gentype-dep, -bs-gentype-source-dir,
  -bs-gentype-bsb-project-root. -bs-project-root now also populates
  GenTypeConfig.project_root.
- Config.t.sources becomes a `string list` (pre-expanded directories)
  instead of raw Ext_json_types, simplifying ModuleResolver.
- Debug.set_item takes a bool-implied item name instead of an
  Ext_json_types.t value.
- Paths.ml: drop get_config_file; read_config is now a thin wrapper
  around build_config.

Rewatch side:
- Replace `GenTypeConfig = serde_json::Value` with a typed struct;
  Deserialize supports both the object and (deprecated) array forms
  of `shims`.
- New Config::get_gentype_args assembles the full -bs-gentype-* flag
  set from the typed gentypeconfig, suffix, dependencies, pre-expanded
  source dirs, and workspace root.
- compile.rs: when gentype is enabled, walk the package's declared
  source folders (honoring `subdirs: true`) to produce the directory
  list gentype needs for cross-file shim resolution — `package.dirs`
  alone only tracks dirs with .res files.

Generated .gen.tsx / .bs.js output is byte-identical; all existing
gentype and full test suites pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Delete rescript.json filesystem walk from the OCaml compiler

With rewatch always passing -bs-project-root, the fallback walk in
Ext_path.find_root_filename was only reachable from direct bsc
invocations. Move that walk to the Node.js wrapper (cli/bsc.js), which
auto-injects -bs-project-root when absent, and make Ext_path.package_dir
fail loudly if the flag was never set.

- Ext_path: drop find_root_filename, find_config_dir, fallback_package_dir.
  package_dir () now errors out with a clear message if custom_package_dir
  is None.
- Literals.rescript_json: removed (last reference was the deleted walk).
- cli/bsc.js: walk up from cwd for rescript.json, append
  -bs-project-root <dir> to the args when it isn't already there.

The compiler now has zero code paths that read or locate rescript.json.
Locating the project root is a concern entirely for the build system
(rewatch) or the JS wrapper.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Remove the custom Ext_json parser

Three sites still used compiler/ext/ext_json* after the rescript.json
removal: gentype's .sourcedirs.json reader, reanalyze's cmt-scan
reader, and the parser's own ounit test suite. Each is handled
separately so the custom lexer-based parser can be deleted outright.

Gentype:
- Add -bs-gentype-dep-path <name>=<path> flag. rewatch emits one per
  dependency, resolved through the packages map it already holds.
- GenTypeConfig.Config.t gains a dep_paths Hashtbl populated from the
  flags. ModuleResolver.sourcedirs_to_map now reads from config.sources
  + config.dep_paths instead of opening <bsb_project_root>/lib/bs/
  .sourcedirs.json. The missing-.sourcedirs.json warning and the
  read_dirs_from_config fallback are gone — rewatch supplies both
  pieces directly.

Reanalyze:
- Delete dead readSourceDirs and readDirsFromConfig (no callers).
- Port readCmtScan from Ext_json_parse to jsonlib (Json.parse), which
  reanalyze already depends on for its rescript.json reading.
- Drop `ext` from reanalyze's dune libraries.

Parser:
- Delete ext_json.ml/mli, ext_json_parse.mll/mli, ext_json_noloc.ml/
  mli, ext_json_types.ml.
- Delete the ocamllex rule in compiler/ext/dune.
- Delete ounit_ext_json_tests.ml and its registration in
  ounit_tests_main.ml.

Net: -1006 lines. No code anywhere in the repo parses JSON with the
custom lexer now; jsonlib is the only JSON parser left.

Generated .gen.tsx / .bs.js output is byte-identical; make test,
test-gentype, and test-rewatch all pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Delete unused ext modules

Three small modules in compiler/ext had no callers anywhere in the
repo (neither in production code, nor in tests):

- ext_color: unused color/ANSI helpers
- ext_position: unused Lexing.position utilities
- ext_string_array: unused string-array helpers

Removed alongside the Ext_json cleanup. Build and tests unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Preserve package-specs.module fallback for gentype

When gentypeconfig.module is omitted, the old JSON-reading path fell
back to package-specs.module (object form) before defaulting to
ESModule. My flag-based rewrite dropped that fallback, so a project
with "package-specs": { "module": "commonjs" } but no explicit
gentypeconfig.module would suddenly emit ESModule-style imports —
breaking downstream TypeScript consumption.

Replay the old precedence in rewatch when assembling
-bs-gentype-module:

1. gentypeconfig.module if set
2. else package-specs.module when package-specs is a single object
3. else leave the flag off (bsc applies its ESModule default, same as
   before)

Array-form package-specs still doesn't feed the fallback — that
matches the old code's Obj-only pattern exactly. Added two tests
covering both paths.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Preserve list-order and dep-path-order semantics from the old code

Two small alignments found during a detailed behavioral review. Each
only matters in a pathological collision case (two dirs / deps with
the same module name), but worth getting right since this PR promises
byte-identical behavior.

1. bs_dependencies / sources: old code accumulated these with a
   prepend into a ref and returned the ref unreversed, so iteration
   proceeded in reverse-input order. My new code did List.rev and
   flipped the order, which would swap the last-write-wins winner in
   ModuleResolver's file_map on collision. Drop the List.rev.

2. dep_paths: I was sorting alphabetically in rewatch. The old code
   iterated the .sourcedirs.json "pkgs" array in order. Remove the
   sort and pass the caller's order through unchanged.

Not preserved (agreed with reviewer that this was a bug): the old
code's namespace field was populated from the cmt filename only when
rescript.json had {"namespace": true} literally. For string-valued
namespaces like {"namespace": "Custom"}, the old pattern match fell
through and returned None — gentype then emitted imports like
./Module pointing at files actually named Module-Custom.bs.js. The
new code (always use the cmt-derived namespace) is correct.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Cache gentype source-dir walk on Package

collect_gentype_source_dirs was being called from compiler_args — i.e.
once per source file per build. The result is per-package and doesn't
change during a build, so the redundant walks were pure waste on
gentype-enabled projects with many files.

Move the walk into package discovery (packages::make), cache it on a
new Package.gentype_dirs field, and only compute it for packages that
have a gentype_config. compiler_args now just borrows the cached
slice.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Derive Deserialize for GenTypeModule* enums

The manual Deserialize impls were reinventing what serde can do with
#[serde(rename = "...")] on each variant — the same pattern already in
use for PackageModule in this file. Drop the hand-written impls in
favor of derive; serde's default error message for unknown variants
("unknown variant `foo`, expected one of ...") is good enough.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Rename Ext_path.custom_package_dir to project_root

\"Custom\" was a holdover from when there were two sources for the
package dir (CLI flag vs filesystem walk fallback) — \"custom\" meant
\"CLI-set\" in contrast to the walked-up default. Now that the walk is
gone, there's only one source and \"custom\" adds no information.

Rename the ref to [project_root] so it matches the CLI flag name
[-bs-project-root] directly. The getter stays [package_dir ()] since
that's the established term used by [lam_compile_main] and
[js_packages_info].

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Drop misplaced comment in GenTypeConfig.build_config

The comment was positioned above the record literal but only applied
to two fields inside it, making it read as if it documented the whole
construction. The invariant it described (kept via commit history) is
better tracked in git blame than inlined at the call site.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Add changelog entry for rescript.json read removal

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Trim changelog entry

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>