feat: refactor vector storage configuration and implement LanceDB

- Removed the Qdrant vector configuration file (`qdrant.ts`) and its associated logic.
- Introduced a new LanceDB vector configuration file (`lance.ts`) to support vector storage and similarity search using LanceDB.
- Implemented comprehensive settings for LanceDB, including memory management, embedding generation, and advanced filtering capabilities.
- Added functions for generating embeddings, validating filters, and performing storage operations with detailed logging.
- Created new JSON files for crawler statistics and session pool state to track request metrics and session management.
- Added a request queue JSON file to manage requests with unique identifiers and metadata.

chore: update storage and request queue structures

- Created `SDK_CRAWLER_STATISTICS_0.json` to log crawler performance metrics.
- Created `SDK_SESSION_POOL_STATE.json` to manage session states and usability.
- Created a request queue JSON file to handle requests with relevant metadata and error tracking.