Commit Graph

196 Commits

Author SHA1 Message Date
78bc110685 search-index-fix2
Some checks failed
Deploy on push / deploy (push) Failing after 5m42s
2025-09-10 12:39:00 +03:00
6817fb6436 search-index-reload 2025-09-10 12:29:59 +03:00
b70901f8f7 ## [0.9.19] - 2025-09-01
Some checks failed
Deploy on push / deploy (push) Failing after 5m57s
### 🚀 ML Models Runtime Preloading
- **🔧 models loading**: Перенесена предзагрузка ML моделей из Docker build в runtime startup
  - Убрана предзагрузка из `Dockerfile` - модели теперь загружаются после монтирования `/dump` папки
  - Добавлена async функция `preload_models()` в `services/search.py` для фоновой загрузки
  - Интеграция предзагрузки в `lifespan` функцию `main.py`
  - Использование `asyncio.run_in_executor()` для неблокирующей загрузки моделей
  - Исправлена проблема с недоступностью `/dump` папки во время сборки Docker образа
2025-09-01 16:38:23 +03:00
9daade05c0 model-path-fix
Some checks failed
Deploy on push / deploy (push) Failing after 1m14s
2025-09-01 16:10:10 +03:00
a1e4d0d391 search-restore-2
Some checks failed
Deploy on push / deploy (push) Failing after 9s
2025-09-01 15:19:05 +03:00
4489d25913 ## [0.9.18] - 2025-01-09
Some checks failed
Deploy on push / deploy (push) Failing after 1m34s
### 🔍 Search System Redis Storage
- **💾 Redis-based vector index storage**: Переключились обратно на Redis для хранения векторного индекса
  - Заменили файловое хранение в `/dump` на Redis ключи для надежности
  - Исправлена проблема с правами доступа на `/dump` папку на сервере
  - Векторный индекс теперь сохраняется по ключам `search_index:{name}:data` и `search_index:{name}:metadata`
- **🛠️ Improved reliability**: Убрали зависимость от файловой системы для критичных данных
- ** Better performance**: Redis обеспечивает более быстрый доступ к индексу
- **🔧 Technical changes**:
  - Заменили `save_index_to_file()` на `save_index_to_redis()`
  - Заменили `load_index_from_file()` на `load_index_from_redis()`
  - Обновили автосохранение для использования Redis вместо файлов
  - Удалили неиспользуемые импорты (`gzip`, `pathlib`, `cast`)
2025-09-01 15:09:36 +03:00
db3dafa569 embedding-search
Some checks failed
Deploy on push / deploy (push) Failing after 22m28s
2025-08-31 19:20:43 +03:00
7325cdc5f5 [0.9.15] - 2025-08-30
All checks were successful
Deploy on push / deploy (push) Successful in 5m42s
### 🔧 Fixed
- **🧾 Database Table Creation**: Унифицирован подход к созданию таблиц БД между продакшеном и тестами
  - Исправлена ошибка "no such table: author" в тестах
  - Обновлена функция `create_all_tables()` в `storage/schema.py` для использования стандартного SQLAlchemy подхода
  - Улучшены фикстуры тестов с принудительным импортом всех ORM моделей
  - Добавлена детальная диагностика создания таблиц в тестах
  - Добавлены fallback механизмы для создания таблиц в проблемных окружениях

### 🧪 Testing
- Все RBAC тесты теперь проходят успешно
- Исправлены фикстуры `test_engine`, `db_session` и `test_session_factory`
- Добавлены функции `ensure_all_tables_exist()` и `ensure_all_models_imported()` для диагностики

### 📝 Technical Details
- Заменен подход `create_table_if_not_exists()` на стандартный `Base.metadata.create_all()`
- Улучшена обработка ошибок при создании таблиц
- Добавлена проверка регистрации всех критических таблиц в metadata
2025-08-30 22:20:58 +03:00
98f625ec0d index-metric
All checks were successful
Deploy on push / deploy (push) Successful in 5m45s
2025-08-30 21:20:01 +03:00
906c9bbdf4 search-index-metric 2025-08-30 21:18:48 +03:00
c9e1d9d878 muvera-index-fix
Some checks failed
Deploy on push / deploy (push) Has been cancelled
2025-08-30 20:41:13 +03:00
00a866876c search-wrapper
Some checks failed
Deploy on push / deploy (push) Failing after 4m31s
2025-08-23 14:08:34 +03:00
b4f683a7cc fmt
Some checks failed
Deploy on push / deploy (push) Failing after 36s
2025-08-23 10:47:52 +03:00
9a2b792f08 refactored
Some checks failed
Deploy on push / deploy (push) Failing after 6s
2025-08-17 17:56:31 +03:00
e78e12eeee circular-fix
Some checks failed
Deploy on push / deploy (push) Failing after 17s
2025-08-17 16:33:54 +03:00
c80f3efc77 mypy-fixed
Some checks failed
Deploy on push / deploy (push) Failing after 5s
2025-07-31 19:27:58 +03:00
809bda2b56 search-fix, devstart-fix, cache-fix, logs-less
Some checks failed
Deploy on push / deploy (push) Failing after 5s
2025-07-31 19:13:23 +03:00
e7230ba63c tests-passed 2025-07-31 18:55:59 +03:00
7585dae0ab less search logs 2025-07-01 12:18:40 +03:00
8a5f4a2421 maintainance 2025-06-16 20:20:23 +03:00
0375939e73 hardcopy-search-service-code
All checks were successful
Deploy on push / deploy (push) Successful in 6s
2025-06-03 02:10:08 +03:00
1329aee1f1 search-combined
All checks were successful
Deploy on push / deploy (push) Successful in 6s
2025-06-03 02:00:44 +03:00
903065fdb3 search-debug
Some checks failed
Deploy on push / type-check (push) Failing after 6s
Deploy on push / deploy (push) Has been skipped
2025-06-02 22:40:10 +03:00
21d28a0d8b token-storage-refactored
Some checks failed
Deploy on push / type-check (push) Failing after 8s
Deploy on push / deploy (push) Has been skipped
2025-06-02 21:50:58 +03:00
3327976586 Improve topic sorting: add popular sorting by publications and authors count 2025-06-02 02:56:11 +03:00
4070f4fcde linted+fmt
All checks were successful
Deploy on push / deploy (push) Successful in 6s
2025-05-29 12:37:39 +03:00
ab39b534fe auth fixes, search connected 2025-05-22 04:34:30 +03:00
Stepan Vladovskiy
c344fcee2d refactoring(search.py): logs for search-combine and search-authors are equal
All checks were successful
Deploy on push / deploy (push) Successful in 6s
2025-05-02 18:28:06 -03:00
Stepan Vladovskiy
a1a61a6731 feat: follow same logic as search shouts for authors. Store them to Reddis cache + pagination
All checks were successful
Deploy on push / deploy (push) Successful in 41s
2025-05-02 18:17:05 -03:00
Stepan Vladovskiy
3782a9dffb fix(search.py, author.py): small fixes for start. logger import fails
All checks were successful
Deploy on push / deploy (push) Successful in 40s
2025-04-29 17:50:51 -03:00
Stepan Vladovskiy
93c00b3dd1 feat(author.py):addresolver for searching authors by text
All checks were successful
Deploy on push / deploy (push) Successful in 1m15s
2025-04-29 17:45:37 -03:00
Stepan Vladovskiy
fac43e5997 refact(search,reader): withput any kind of sorting
All checks were successful
Deploy on push / deploy (push) Successful in 42s
2025-04-24 21:00:41 -03:00
Stepan Vladovskiy
e7facf8d87 style(search.py): with indexing message
All checks were successful
Deploy on push / deploy (push) Successful in 42s
2025-04-24 18:45:00 -03:00
Stepan Vladovskiy
3062a2b7de refactor(search.py): with checking titles without bodies for not re indexing them every startup
All checks were successful
Deploy on push / deploy (push) Successful in 42s
2025-04-24 14:58:14 -03:00
Stepan Vladovskiy
c0406dbbf2 refac(search.py): without logger and rm dublicated def search-text
All checks were successful
Deploy on push / deploy (push) Successful in 44s
2025-04-24 14:18:14 -03:00
Stepan Vladovskiy
5425dbf832 refactor(search.py): simplify def search 2025-04-24 13:46:58 -03:00
Stepan Vladovskiy
a10db2d38a feat(search.py): combined search on shouts tittles and bodys 2025-04-24 13:35:36 -03:00
Stepan Vladovskiy
11654dba68 feat: with three separate endpoints
All checks were successful
Deploy on push / deploy (push) Successful in 5s
2025-04-23 18:24:00 -03:00
Stepan Vladovskiy
4d965fb27b feat(search.py): separate indexing of Shout Title, shout Body and Authors
All checks were successful
Deploy on push / deploy (push) Successful in 39s
2025-04-20 19:22:08 -03:00
Stepan Vladovskiy
106222b0e0 debug: without debug logging. clean
All checks were successful
Deploy on push / deploy (push) Successful in 1m27s
2025-04-07 11:41:48 -03:00
Stepan Vladovskiy
78326047bf fix(reader.py): change sorting and answer on querys
All checks were successful
Deploy on push / deploy (push) Successful in 50s
2025-04-03 13:20:18 -03:00
Stepan Vladovskiy
bc4ec79240 fix(search.py): store all results in cash not only first offset
All checks were successful
Deploy on push / deploy (push) Successful in 52s
2025-04-03 13:10:53 -03:00
Stepan Vladovskiy
a0db5707c4 feat: add cash for storing searchresalts and hold them for working pagination. Now we are have offset for use on frontend
All checks were successful
Deploy on push / deploy (push) Successful in 51s
2025-04-01 16:01:09 -03:00
Stepan Vladovskiy
e405fb527b refactor(search.py): moved to use one table docs for embdings and docs store
All checks were successful
Deploy on push / deploy (push) Successful in 50s
2025-03-25 16:42:44 -03:00
Stepan Vladovskiy
7f36f93d92 feat(search.py): detects both missing documents and null embeddings
All checks were successful
Deploy on push / deploy (push) Successful in 1m32s
2025-03-25 15:18:29 -03:00
Stepan Vladovskiy
f089a32394 debug(search.py): with more logs when check sync of indexing
All checks were successful
Deploy on push / deploy (push) Successful in 1m3s
2025-03-25 14:44:05 -03:00
Stepan Vladovskiy
1fd623a660 feat: with index sync endpoints configs
All checks were successful
Deploy on push / deploy (push) Successful in 56s
2025-03-25 13:31:45 -03:00
Stepan Vladovskiy
077cb46482 debug: server.py -> threds 1 , search.py -> add 3 times reconect
All checks were successful
Deploy on push / deploy (push) Successful in 49s
2025-03-24 20:16:07 -03:00
Stepan Vladovskiy
60a13a9097 refactor(search.py): moved initialization logic in search-txtai instance
All checks were successful
Deploy on push / deploy (push) Successful in 55s
2025-03-24 19:47:02 -03:00
Stepan Vladovskiy
316375bf18 debug(search.py): encrease batch size for bulk indexing
All checks were successful
Deploy on push / deploy (push) Successful in 1m1s
2025-03-21 17:56:54 -03:00