Не сканирует после развертывания Версия 0.47-devel

2 years 6 months ago - 2 years 6 months ago #3234 by LAD75
LAD75 created the topic: Не сканирует после развертывания Версия 0.47-devel
поставил по гайду, переключил на mysql, задал каталог в виде сим-линка на каталог от торрента (для обновления).
файлы в zip-архивах.
Операционная система Ubuntu Linux 18.04.6
до этого все работало как часы, но сегодня обновил всю систему. была 16 что-то там

запускаю скан, получаю ошибку.
не знаю куда копать, т.к. не понимаю ничего в этих сообщениях :(

помогите победить
root@home:/opt/sopds# python3 manage.py sopds_scanner scan --verbose >>1
2021-10-07 20:26:41,834 INFO      ***** Starting sopds-scan...
2021-10-07 20:26:41,834 DEBUG    OPTIONS SET
2021-10-07 20:26:41,837 DEBUG    root_lib = /opt/sopds/books
2021-10-07 20:26:41,840 DEBUG    fb2toepub =
2021-10-07 20:26:41,843 DEBUG    fb2tomobi =
2021-10-07 20:26:41,845 DEBUG    temp_dir = /opt/sopds/tmp
2021-10-07 20:26:41,848 INFO     FB2SAX = True
2021-10-07 20:26:41,859 DEBUG    Skip ZIP archive fb2-649000-650999.zip. Already scanned.
2021-10-07 20:26:41,865 DEBUG    Skip ZIP archive fb2-615000-617999.zip. Already scanned.
2021-10-07 20:26:41,871 DEBUG    Skip ZIP archive fb2-161831-166042.zip. Already scanned.
2021-10-07 20:26:41,877 DEBUG    Skip ZIP archive fb2-579000-581999.zip. Already scanned.
2021-10-07 20:26:41,983 DEBUG    Start process ZIP file = /opt/sopds/books/fb2-147519-153549.zip book file = 147519.fb2
2021-10-07 20:26:42,005 DEBUG    Attempt to add book fb2-147519-153549.zip/147519.fb2
/usr/local/lib/python3.6/dist-packages/django/db/backends/mysql/base.py:71: Warning: (1366, "Incorrect string value: '\\xD0\\x94\\xD0\\xBE\\xD1\\x80...' for column 'title' at row 1")
  return self.cursor.execute(query, args)
/usr/local/lib/python3.6/dist-packages/django/db/backends/mysql/base.py:71: Warning: (1366, "Incorrect string value: '\\xD0\\x94\\xD0\\x9E\\xD0\\xA0...' for column 'search_title' at row 1")
  return self.cursor.execute(query, args)
/usr/local/lib/python3.6/dist-packages/django/db/backends/mysql/base.py:71: Warning: (1366, "Incorrect string value: '\\xD0\\x90\\xD0\\xBD\\xD0\\xB4...' for column 'annotation' at row 1")
  return self.cursor.execute(query, args)
2021-10-07 20:26:42,016 DEBUG    Book fb2-147519-153549.zip/147519.fb2 Added ok.
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/mysql/base.py", line 71, in execute
    return self.cursor.execute(query, args)
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 250, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler
    raise errorvalue
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 247, in execute
    res = self._query(query)
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 411, in _query
    rowcount = self._do_query(q)
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 374, in _do_query
    db.query(q)
  File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 292, in query
    _mysql.connection.query(self, query)
_mysql_exceptions.OperationalError: (1267, "Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation '='")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.6/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.6/dist-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.6/dist-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.6/dist-packages/django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "/opt/sopds/opds_catalog/management/commands/sopds_scanner.py", line 57, in handle
    self.scan()
  File "/opt/sopds/opds_catalog/management/commands/sopds_scanner.py", line 80, in scan
    scanner.scan_all()
  File "/opt/sopds/opds_catalog/sopdscan.py", line 104, in scan_all
    self.processzip(name,full_path,file)
  File "/opt/sopds/opds_catalog/sopdscan.py", line 191, in processzip
    self.processfile(n,file,bookfile,cat,opdsdb.CAT_ZIP,file_size)
  File "/opt/sopds/opds_catalog/sopdscan.py", line 240, in processfile
    author=opdsdb.addauthor(author_name)
  File "/opt/sopds/opds_catalog/opdsdb.py", line 247, in addauthor
    'lang_code':getlangcode(full_name)})
  File "/usr/local/lib/python3.6/dist-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py", line 486, in get_or_create
    return self.get(**lookup), False
  File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py", line 393, in get
    num = len(clone)
  File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py", line 250, in __len__
    self._fetch_all()
  File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py", line 1186, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py", line 54, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/usr/local/lib/python3.6/dist-packages/django/db/models/sql/compiler.py", line 1098, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py", line 100, in execute
    return super().execute(sql, params)
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.6/dist-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/mysql/base.py", line 71, in execute
    return self.cursor.execute(query, args)
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 250, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler
    raise errorvalue
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 247, in execute
    res = self._query(query)
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 411, in _query
    rowcount = self._do_query(q)
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 374, in _do_query
    db.query(q)
  File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 292, in query
    _mysql.connection.query(self, query)
django.db.utils.OperationalError: (1267, "Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation '='")


м.б. что-то с кодовой страницей?
у меня все поля в БД вот в таком виде "??????? ?????????? ? ?????"


переделал кодировку БД на win1251
пересоздал
запустил сканирование:
root@home:/opt/sopds# python3 manage.py sopds_scanner start --verbose
Startup scheduled book-scan (min=0, hour=0,12, day_of_week=*, day=*).
2021-10-07 22:19:26,788 INFO Adding job tentatively -- it will be properly scheduled when the scheduler starts
2021-10-07 22:19:26,789 INFO Adding job tentatively -- it will be properly scheduled when the scheduler starts
Quit the server with CONTROL-C.
2021-10-07 22:19:26,790 INFO Added job "Command.scan" to job store "default"
2021-10-07 22:19:26,790 INFO Added job "Command.check_settings" to job store "default"
2021-10-07 22:19:26,790 INFO Scheduler started
2021-10-07 22:19:26,790 DEBUG Looking for jobs to run
2021-10-07 22:19:26,790 DEBUG Next wakeup is due at 2021-10-07 22:20:00+03:00 (in 33.209216 seconds)
2021-10-07 22:20:00,000 DEBUG Looking for jobs to run
2021-10-07 22:20:00,004 DEBUG Next wakeup is due at 2021-10-07 22:30:00+03:00 (in 599.999342 seconds)
2021-10-07 22:20:00,002 INFO Running job "Command.check_settings (trigger: cron[minute='*/10'], next run at: 2021-10-07 22:30:00 MSK)" (scheduled at 2021-10-07 22:20:00+03:00)
2021-10-07 22:20:00,018 INFO Job "Command.check_settings (trigger: cron[minute='*/10'], next run at: 2021-10-07 22:30:00 MSK)" executed successfully


собственно, получается, что сканер не видит книги? как можно понять причину?
Last Edit: 2 years 6 months ago by LAD75.

Please Войти or Create an account to join the conversation.

  • LAD75
  • LAD75's Avatar Topic Author
  • Offline
  • Новый участник
  • Новый участник
More
2 years 6 months ago #3235 by LAD75
LAD75 replied the topic: Не сканирует после развертывания Версия 0.47-devel
переделал все с нуля еще раз.
появляется ошибка. имена файлов корректные. внутри в архивах тоже.
но при сканировании выдает ошибку. с какой кодировкой создавать БД для библиотеки?

Warning: Spoiler! [ Click to expand ]

что не так? не понимаю (((((((

Please Войти or Create an account to join the conversation.

  • LAD75
  • LAD75's Avatar Topic Author
  • Offline
  • Новый участник
  • Новый участник
More
2 years 6 months ago #3236 by LAD75
LAD75 replied the topic: Не сканирует после развертывания Версия 0.47-devel
после очередного создания БД по гайду
coderoad.ru/3029321/%D0%A3%D1%81%D1%82%D...A%D0%B0-%D0%B2-mysql

в части: CHARACTER SET utf8 COLLATE utf8_unicode_ci;
стартануло сканирование каталога


странно, что изначально скрипт инициализации не создал БД с нужными параметрами

Please Войти or Create an account to join the conversation.

  • LAD75
  • LAD75's Avatar Topic Author
  • Offline
  • Новый участник
  • Новый участник
More
Time to create page: 0.313 seconds