Don't rule out running from the command line

Fix byte string error for KFX

.github/workflows/Format.yaml

@@ -0,0 +1,39 @@
+name: Python code format
+on: 
+  push:
+    branches: master
+jobs:
+  Format:
+    if: "contains(github.event.head_commit.message, '!format')"
+    runs-on: ubuntu-20.04
+    strategy:
+      fail-fast: false
+    steps:
+      - uses: actions/checkout@main
+      - name: Set up Python
+        uses: actions/setup-python@main
+        with:
+          python-version: 3.x
+      - uses: actions/cache@main
+        with:
+          path: ~/.cache/pip
+          key: ${{ runner.os }}-pip-format
+          restore-keys: |
+            ${{ runner.os }}-pip-format
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install autopep8 pycodestyle
+      - name: Format by autopep8 then Push
+        env:
+          GIT_EMAIL: github-actions[bot]@users.noreply.github.com
+          GIT_ACTOR: github-actions[bot]
+        run: |
+          export HASH_SHORT=$(git rev-parse --short HEAD)
+          git checkout -b format--${HASH_SHORT}
+          git config --global user.email $GIT_EMAIL
+          git config --global user.name  $GIT_ACTOR
+          python -m autopep8 --in-place --aggressive --aggressive --experimental -r ./
+          git add -A
+          git commit -m 'Format by autopep8' -m From: -m $(git rev-parse HEAD)
+          git push --set-upstream origin format--${HASH_SHORT}

.github/workflows/Lint.yaml

@@ -0,0 +1,26 @@
+name: Python code review
+on: [push, pull_request]
+jobs:
+  Test:
+    runs-on: ubuntu-20.04
+    strategy:
+      fail-fast: false
+    steps:
+      - uses: actions/checkout@main
+      - name: Set up Python
+        uses: actions/setup-python@main
+        with:
+          python-version: 3.x
+      - uses: actions/cache@main
+        with:
+          path: ~/.cache/pip
+          key: ${{ runner.os }}-pip-lint
+          restore-keys: |
+            ${{ runner.os }}-pip-lint
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install flake8
+      - name: Lint with flake8
+        run: |
+          python -m flake8 . --builtins=_,I --ignore=E501 --count --benchmark --show-source --statistics

.github/workflows/Python_tests.yml

@@ -1,35 +0,0 @@
-name: Python_tests
-on: [push, pull_request]
-jobs:
-  Python_tests:
-    runs-on: ${{ matrix.os }}
-    strategy:
-      fail-fast: false
-      matrix:
-        os: [macos-latest]
-        python-version: [2.7, 3.8]
-    steps:
-      - uses: actions/checkout@v2
-      - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v1
-        with:
-          python-version: ${{ matrix.python-version }}
-      - name: Install dependencies
-        run: |
-          python -m pip install --upgrade pip
-          pip install flake8 pytest  # -r requirements.txt
-      #- name: Check formatting with black
-      #  if: matrix.python-version == '3.8'
-      #  run: |
-      #     pip install black
-      #     black --check .
-      - name: Lint with flake8
-        run: |
-          # stop the build if there are Python syntax errors or undefined names
-          flake8 . --builtins=_,I --count --select=E9,F63,F7,F82 --show-source --statistics
-          # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
-          flake8 . --builtins=_,I --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
-      #- name: Test with pytest
-      #  run: pytest
-      #- name: Run doctests with pytest
-      #  run: pytest --doctest-modules

DeDRM_plugin/DeDRM_Adobe Digital Editions Key_Help.htm

@@ -38,7 +38,7 @@ li {margin-top: 0.5em}
 
 <h3>Renaming Keys:</h3>
 
-<p>On the right-hand side of the plugin’s customization dialog, you will see a button with an icon that looks like a sheet of paper. Clicking this button will promt you to enter a new name for the highlighted key in the list. Enter the new name for the encryption key and click the OK button to use the new name, or Cancel to revert to the old name..</p>
+<p>On the right-hand side of the plugin’s customization dialog, you will see a button with an icon that looks like a sheet of paper. Clicking this button will prompt you to enter a new name for the highlighted key in the list. Enter the new name for the encryption key and click the OK button to use the new name, or Cancel to revert to the old name..</p>
 
 <h3>Exporting Keys:</h3>
 

DeDRM_plugin/DeDRM_Barnes and Noble Key_Help.htm

@@ -46,7 +46,7 @@ li {margin-top: 0.5em}
 
 <h3>Renaming Keys:</h3>
 
-<p>On the right-hand side of the plugin’s customization dialog, you will see a button with an icon that looks like a sheet of paper. Clicking this button will promt you to enter a new name for the highlighted key in the list. Enter the new name for the encryption key and click the OK button to use the new name, or Cancel to revert to the old name..</p>
+<p>On the right-hand side of the plugin’s customization dialog, you will see a button with an icon that looks like a sheet of paper. Clicking this button will prompt you to enter a new name for the highlighted key in the list. Enter the new name for the encryption key and click the OK button to use the new name, or Cancel to revert to the old name..</p>
 
 <h3>Exporting Keys:</h3>
 
@@ -56,7 +56,7 @@ li {margin-top: 0.5em}
 
 <p>At the bottom-left of the plugin’s customization dialog, you will see a button labeled "Import Existing Keyfiles". Use this button to import existing ‘.b64’ key files. Key files might come from being exported from this or older plugins, or may have been generated using the original i♥cabbages script, or you may have made it by following the instructions above.</p>
 
-<p>Once done creating/deleting/renaming/importing decryption keys, click Close to exit the customization dialogue. Your changes wil only be saved permanently when you click OK in the main configuration dialog.</p>
+<p>Once done creating/deleting/renaming/importing decryption keys, click Close to exit the customization dialogue. Your changes will only be saved permanently when you click OK in the main configuration dialog.</p>
 
 <h3>NOOK Study</h3>
 <p>Books downloaded through NOOK Study may or may not use the key found using the above method. If a book is not decrypted successfully with any of the keys, the plugin will attempt to recover keys from the NOOK Study log file and use them.</p>

DeDRM_plugin/DeDRM_EInk Kindle Serial Number_Help.htm

@@ -36,7 +36,7 @@ li {margin-top: 0.5em}
 
 <p>On the right-hand side of the plugin’s customization dialog, you will see a button with an icon that looks like a red "X". Clicking this button will delete the highlighted Kindle serial number from the list. You will be prompted once to be sure that’s what you truly mean to do. Once gone, it’s permanently gone.</p>
 
-<p>Once done creating/deleting serial numbers, click Close to exit the customization dialogue. Your changes wil only be saved permanently when you click OK in the main configuration dialog.</p>
+<p>Once done creating/deleting serial numbers, click Close to exit the customization dialogue. Your changes will only be saved permanently when you click OK in the main configuration dialog.</p>
 
 </body>
 

DeDRM_plugin/DeDRM_Kindle for Mac and PC Key_Help.htm

@@ -38,7 +38,7 @@ li {margin-top: 0.5em}
 
 <h3>Renaming Keys:</h3>
 
-<p>On the right-hand side of the plugin’s customization dialog, you will see a button with an icon that looks like a sheet of paper. Clicking this button will promt you to enter a new name for the highlighted key in the list. Enter the new name for the encryption key and click the OK button to use the new name, or Cancel to revert to the old name..</p>
+<p>On the right-hand side of the plugin’s customization dialog, you will see a button with an icon that looks like a sheet of paper. Clicking this button will prompt you to enter a new name for the highlighted key in the list. Enter the new name for the encryption key and click the OK button to use the new name, or Cancel to revert to the old name..</p>
 
 <h3>Exporting Keys:</h3>
 
@@ -46,7 +46,7 @@ li {margin-top: 0.5em}
 
 <h3>Linux Users: WINEPREFIX</h3>
 
-<p>Under the list of keys, Linux users will see a text field labeled "WINEPREFIX". If you are use Kindle for PC under Wine, and your wine installation containing Kindle for PC isn't the default Wine installation, you may enter the full path to the correct Wine installation here. Leave blank if you are unsure.</p>
+<p>Under the list of keys, Linux users will see a text field labeled "WINEPREFIX". If you are using the Kindle for PC under Wine, and your wine installation containing Kindle for PC isn't the default Wine installation, you may enter the full path to the correct Wine installation here. Leave blank if you are unsure.</p>
 
 <h3>Importing Existing Keyfiles:</h3>
 

DeDRM_plugin/__init__.py

@@ -69,6 +69,7 @@ __docformat__ = 'restructuredtext en'
 #   6.6.3 - More cleanup of kindle book names and start of support for .kinf2018
 #   6.7.0 - Handle new library in calibre.
 #   6.8.0 - Full support for .kinf2018 and new KFX encryption (Kindle for PC/Mac 2.5+)
+#   6.8.1 - Kindle key fix for Mac OS X Big Syr
 #   7.0.0 - Switched to Python 3 for calibre 5.0. Thanks to all who comtibuted
 
 """

DeDRM_plugin/androidkindlekey.py

@@ -2,7 +2,7 @@
 # -*- coding: utf-8 -*-
 
 # androidkindlekey.py
-# Copyright © 2010-20 by Thom, Apprentice  et al.
+# Copyright © 2010-20 by Thom, Apprentice Harper et al.
 
 # Revision history:
 #  1.0   - AmazonSecureStorage.xml decryption to serial number
@@ -30,10 +30,7 @@ import tempfile
 import zlib
 import tarfile
 from hashlib import md5
-try:
+from io import BytesIO
-    from cStringIO import StringIO
-except ImportError:
-    from io import BytesIO as StringIO
 from binascii import a2b_hex, b2a_hex
 
 # Routines common to Mac and PC
@@ -116,7 +113,7 @@ class AndroidObfuscation(object):
         cipher = self._get_cipher()
         padding = len(self.key) - len(plaintext) % len(self.key)
         plaintext += chr(padding) * padding
-        return b2a_hex(cipher.encrypt(plaintext))
+        return b2a_hex(cipher.encrypt(plaintext.encode('utf-8')))
 
     def decrypt(self, ciphertext):
         cipher = self._get_cipher()
@@ -182,7 +179,7 @@ def get_serials1(path=STORAGE1):
         obfuscation = AndroidObfuscation()
 
     def get_value(key):
-        encrypted_key = obfuscation.encrypt(a2b_hex(key))
+        encrypted_key = obfuscation.encrypt(key)
         encrypted_value = storage.get(encrypted_key)
         if encrypted_value:
             return obfuscation.decrypt(encrypted_value)
@@ -198,6 +195,7 @@ def get_serials1(path=STORAGE1):
     try:
         tokens = set(get_value('kindle.account.tokens').split(','))
     except:
+        sys.stderr.write('cannot get kindle account tokens\n')
         return []
 
     serials = []
@@ -251,8 +249,8 @@ def get_serials2(path=STORAGE2):
 
     serials = []
     for x in dsns:
+        serials.append(x)
         for y in tokens:
-            serials.append(x)
             serials.append(y)
             serials.append(x+y)
     return serials
@@ -277,7 +275,7 @@ def get_serials(path=STORAGE):
         read = open(path, 'rb')
         head = read.read(24)
         if head[:14] == b'ANDROID BACKUP':
-            output = StringIO(zlib.decompress(read.read()))
+            output = BytesIO(zlib.decompress(read.read()))
     except Exception:
         pass
     finally:
@@ -314,7 +312,7 @@ def getkey(outfile, inpath):
     if len(keys) > 0:
         with open(outfile, 'w') as keyfileout:
             for key in keys:
-                keyfileout.write(b2a_hex(key))
+                keyfileout.write(key)
                 keyfileout.write("\n")
         return True
     return False
@@ -391,7 +389,7 @@ def gui_main():
         import tkinter.filedialog
     except:
         print("tkinter not installed")
-        return cli_main()
+        return 0
 
     class DecryptionDialog(tkinter.Frame):
         def __init__(self, root):

DeDRM_plugin/epubtest.py

@@ -169,12 +169,12 @@ def getfiledata(file, zi):
 
 def encryption(infile):
     # returns encryption: one of Unencrypted, Adobe, B&N and Unknown
-    encryption = "Unknown"
+    encryption = "Error When Checking."
     try:
         with open(infile,'rb') as infileobject:
             bookdata = infileobject.read(58)
             # Check for Zip
-            if bookdata[0:0+2] == "PK":
+            if bookdata[0:0+2] == b"PK":
                 foundrights = False
                 foundencryption = False
                 inzip = zipfile.ZipFile(infile,'r')
@@ -198,7 +198,10 @@ def encryption(infile):
 
 def main():
     argv=unicode_argv()
-    print(encryption(argv[1]))
+    if len(argv) < 2:
+        print("Give an ePub file as a parameter.")
+    else:
+        print(encryption(argv[1]))
     return 0
 
 if __name__ == "__main__":

DeDRM_plugin/erdr2pml.py

@@ -200,7 +200,7 @@ class Sectionizer(object):
         self.num_sections, = struct.unpack('>H', self.contents[76:78])
         # Dictionary or normal content (TODO: Not hard-coded)
         if self.header[0x3C:0x3C+8] != ident:
-            if self.header[0x3C:0x3C+8] == "PDctPPrs":
+            if self.header[0x3C:0x3C+8] == b"PDctPPrs":
                 self.bkType = "Dict"
             else:
                 raise ValueError('Invalid file format')
@@ -240,7 +240,7 @@ def sanitizeFileName(name):
 def fixKey(key):
     def fixByte(b):
         return b ^ ((b ^ (b<<1) ^ (b<<2) ^ (b<<3) ^ (b<<4) ^ (b<<5) ^ (b<<6) ^ (b<<7) ^ 0x80) & 0x80)
-    return     "".join([chr(fixByte(ord(a))) for a in key])
+    return bytes([fixByte(a) for a in key])
 
 def deXOR(text, sp, table):
     r=''
@@ -269,13 +269,13 @@ class EreaderProcessor(object):
             raise ValueError('incorrect eReader version (error 2)')
         input = des.decrypt(data[-cookie_size:])
         def unshuff(data, shuf):
-            r = [''] * len(data)
+            r = [0] * len(data)
             j = 0
             for i in range(len(data)):
                 j = (j + shuf) % len(data)
                 r[j] = data[i]
-            assert    len("".join(r)) == len(data)
+            assert len(bytes(r)) == len(data)
-            return "".join(r)
+            return bytes(r)
         r = unshuff(input[0:-8], cookie_shuf)
 
         drm_sub_version = struct.unpack('>H', r[0:2])[0]
@@ -354,7 +354,7 @@ class EreaderProcessor(object):
 
     def getImage(self, i):
         sect = self.section_reader(self.first_image_page + i)
-        name = sect[4:4+32].strip('\0')
+        name = sect[4:4+32].strip(b'\0')
         data = sect[62:]
         return sanitizeFileName(name.decode('windows-1252')), data
 
@@ -404,7 +404,7 @@ class EreaderProcessor(object):
 
     def getText(self):
         des = Des(fixKey(self.content_key))
-        r = ''
+        r = b''
         for i in range(self.num_text_pages):
             logging.debug('get page %d', i)
             r += zlib.decompress(des.decrypt(self.section_reader(1 + i)))
@@ -456,8 +456,7 @@ def cleanPML(pml):
     # Convert special characters to proper PML code.  High ASCII start at (\x80, \a128) and go up to (\xff, \a255)
     pml2 = pml
     for k in range(128,256):
-        badChar = chr(k)
+        pml2 = pml2.replace(bytes([k]), b'\\a%03d' % k)
-        pml2 = pml2.replace(badChar, '\\a%03d' % k)
     return pml2
 
 def decryptBook(infile, outpath, make_pmlz, user_key):
@@ -476,7 +475,7 @@ def decryptBook(infile, outpath, make_pmlz, user_key):
         if not os.path.exists(outdir):
             os.makedirs(outdir)
         print("Decoding File")
-        sect  =Sectionizer(infile, 'PNRdPPrs')
+        sect  =Sectionizer(infile, b'PNRdPPrs')
         er = EreaderProcessor(sect, user_key)
 
         if er.getNumImages() > 0:

DeDRM_plugin/ignobleepub.py

@@ -35,6 +35,7 @@ __version__ = "5.0"
 import sys
 import os
 import traceback
+import base64
 import zlib
 import zipfile
 from zipfile import ZipInfo, ZipFile, ZIP_STORED, ZIP_DEFLATED
@@ -151,7 +152,7 @@ def _load_crypto_libcrypto():
 
         def decrypt(self, data):
             out = create_string_buffer(len(data))
-            iv = ("\x00" * self._blocksize)
+            iv = (b'\x00' * self._blocksize)
             rv = AES_cbc_encrypt(data, out, len(data), self._key, iv, 0)
             if rv == 0:
                 raise IGNOBLEError('AES decryption failed')
@@ -164,7 +165,7 @@ def _load_crypto_pycrypto():
 
     class AES(object):
         def __init__(self, key):
-            self._aes = _AES.new(key, _AES.MODE_CBC, '\x00'*16)
+            self._aes = _AES.new(key, _AES.MODE_CBC, b'\x00'*16)
 
         def decrypt(self, data):
             return self._aes.decrypt(data)
@@ -207,15 +208,15 @@ class Decryptor(object):
     def decompress(self, bytes):
         dc = zlib.decompressobj(-15)
         bytes = dc.decompress(bytes)
-        ex = dc.decompress('Z') + dc.flush()
+        ex = dc.decompress(b'Z') + dc.flush()
         if ex:
             bytes = bytes + ex
         return bytes
 
     def decrypt(self, path, data):
-        if path in self._encrypted:
+        if bytes(path,'utf-8') in self._encrypted:
             data = self._aes.decrypt(data)[16:]
-            data = data[:-ord(data[-1])]
+            data = data[:-data[-1]]
             data = self.decompress(data)
         return data
 
@@ -241,7 +242,7 @@ def ignobleBook(inpath):
 def decryptBook(keyb64, inpath, outpath):
     if AES is None:
         raise IGNOBLEError("PyCrypto or OpenSSL must be installed.")
-    key = keyb64.decode('base64')[:16]
+    key = base64.b64decode(keyb64)[:16]
     aes = AES(key)
     with closing(ZipFile(open(inpath, 'rb'))) as inf:
         namelist = set(inf.namelist())
@@ -259,8 +260,8 @@ def decryptBook(keyb64, inpath, outpath):
             if len(bookkey) != 64:
                 print("{0:s} is not a secure Barnes & Noble ePub.".format(os.path.basename(inpath)))
                 return 1
-            bookkey = aes.decrypt(bookkey.decode('base64'))
+            bookkey = aes.decrypt(base64.b64decode(bookkey))
-            bookkey = bookkey[:-ord(bookkey[-1])]
+            bookkey = bookkey[:-bookkey[-1]]
             encryption = inf.read('META-INF/encryption.xml')
             decryptor = Decryptor(bookkey[-16:], encryption)
             kwds = dict(compression=ZIP_DEFLATED, allowZip64=False)

DeDRM_plugin/ineptepub.py

@@ -311,7 +311,7 @@ def _load_crypto_pycrypto():
             total = 0
             for byte in bytes:
                 total = (total << 8) + byte
-            return long(total)
+            return total
 
         def decrypt(self, data):
             return _PKCS1_v1_5.new(self._rsa).decrypt(data, 172)

DeDRM_plugin/ion.py

@@ -31,7 +31,7 @@ import struct
 from io import BytesIO
 
 from Crypto.Cipher import AES
-from Crypto.Util.py3compat import bchr, bord
+from Crypto.Util.py3compat import bchr
 
 try:
     # lzma library from calibre 4.6.0 or later
@@ -89,7 +89,7 @@ LEN_IS_VAR_LEN = 0xE
 LEN_IS_NULL = 0xF
 
 
-VERSION_MARKER = b"\x01\x00\xEA"
+VERSION_MARKER = [b"\x01", b"\x00", b"\xEA"]
 
 
 # asserts must always raise exceptions for proper functioning
@@ -347,7 +347,7 @@ class BinaryIonParser(object):
         b = self.stream.read(1)
         if len(b) < 1:
             return -1
-        b = bord(b)
+        b = ord(b)
         result = b >> 4
         ln = b & 0xF
 
@@ -372,13 +372,13 @@ class BinaryIonParser(object):
         return result
 
     def readvarint(self):
-        b = bord(self.read())
+        b = ord(self.read())
         negative = ((b & 0x40) != 0)
         result = (b & 0x3F)
 
         i = 0
         while (b & 0x80) == 0 and i < 4:
-            b = bord(self.read())
+            b = ord(self.read())
             result = (result << 7) | (b & 0x7F)
             i += 1
 
@@ -389,12 +389,12 @@ class BinaryIonParser(object):
         return result
 
     def readvaruint(self):
-        b = bord(self.read())
+        b = ord(self.read())
         result = (b & 0x7F)
 
         i = 0
         while (b & 0x80) == 0 and i < 4:
-            b = bord(self.read())
+            b = ord(self.read())
             result = (result << 7) | (b & 0x7F)
             i += 1
 
@@ -414,7 +414,7 @@ class BinaryIonParser(object):
         _assert(self.localremaining <= 8, "Decimal overflow")
 
         signed = False
-        b = [bord(x) for x in self.read(self.localremaining)]
+        b = [ord(x) for x in self.read(self.localremaining)]
         if (b[0] & 0x80) != 0:
             b[0] = b[0] & 0x7F
             signed = True
@@ -579,7 +579,7 @@ class BinaryIonParser(object):
                 _assert(self.valuelen <= 4, "int too long: %d" % self.valuelen)
                 v = 0
                 for i in range(self.valuelen - 1, -1, -1):
-                    v = (v | (bord(self.read()) << (i * 8)))
+                    v = (v | (ord(self.read()) << (i * 8)))
 
                 if self.valuetid == TID_NEGINT:
                     self.value = -v
@@ -649,7 +649,7 @@ class BinaryIonParser(object):
 
         result = ""
         for i in b:
-            result += ("%02x " % bord(i))
+            result += ("%02x " % ord(i))
 
         if len(result) > 0:
             result = result[:-1]
@@ -748,7 +748,8 @@ def pkcs7pad(msg, blocklen):
 def pkcs7unpad(msg, blocklen):
     _assert(len(msg) % blocklen == 0)
 
-    paddinglen = bord(msg[-1])
+    paddinglen = msg[-1]
+
     _assert(paddinglen > 0 and paddinglen <= blocklen, "Incorrect padding - Wrong key")
     _assert(msg[-paddinglen:] == bchr(paddinglen) * paddinglen, "Incorrect padding - Wrong key")
 
@@ -806,7 +807,7 @@ class DrmIonVoucher(object):
     secretkey = b""
 
     def __init__(self, voucherenv, dsn, secret):
-        self.dsn,self.secret = dsn,secret
+        self.dsn, self.secret = dsn, secret
 
         self.lockparams = []
 
@@ -827,7 +828,7 @@ class DrmIonVoucher(object):
 
         sharedsecret = obfuscate(shared.encode('ASCII'), self.version)
 
-        key = hmac.new(sharedsecret, "PIDv3", digestmod=hashlib.sha256).digest()
+        key = hmac.new(sharedsecret, b"PIDv3", digestmod=hashlib.sha256).digest()
         aes = AES.new(key[:32], AES.MODE_CBC, self.cipheriv[:16])
         b = aes.decrypt(self.ciphertext)
         b = pkcs7unpad(b, 16)
@@ -1024,7 +1025,7 @@ class DrmIon(object):
             outpages.write(msg)
             return
 
-        _assert(msg[0] == b"\x00", "LZMA UseFilter not supported")
+        _assert(msg[0] == 0, "LZMA UseFilter not supported")
 
         if calibre_lzma is not None:
             with calibre_lzma.decompress(msg[1:], bufsize=0x1000000) as f:

DeDRM_plugin/kfxdedrm.py

@@ -11,6 +11,10 @@ import shutil
 import zipfile
 
 from io import BytesIO
+try:
+    from ion import DrmIon, DrmIonVoucher
+except:
+    from calibre_plugins.dedrm.ion import DrmIon, DrmIonVoucher
 
 
 __license__ = 'GPL v3'
@@ -27,22 +31,18 @@ class KFXZipBook:
         return (None, None)
 
     def processBook(self, totalpids):
-        try:
-            import ion
-        except:
-            from calibre_plugins.dedrm import ion
         with zipfile.ZipFile(self.infile, 'r') as zf:
             for filename in zf.namelist():
                 with zf.open(filename) as fh:
                     data = fh.read(8)
-                    if data != '\xeaDRMION\xee':
+                    if data != b'\xeaDRMION\xee':
                         continue
                     data += fh.read()
                     if self.voucher is None:
                         self.decrypt_voucher(totalpids)
                     print("Decrypting KFX DRMION: {0}".format(filename))
                     outfile = BytesIO()
-                    ion.DrmIon(BytesIO(data[8:-8]), lambda name: self.voucher).parse(outfile)
+                    DrmIon(BytesIO(data[8:-8]), lambda name: self.voucher).parse(outfile)
                     self.decrypted[filename] = outfile.getvalue()
 
         if not self.decrypted:
@@ -53,11 +53,11 @@ class KFXZipBook:
             for info in zf.infolist():
                 with zf.open(info.filename) as fh:
                     data = fh.read(4)
-                    if data != '\xe0\x01\x00\xea':
+                    if data != b'\xe0\x01\x00\xea':
                         continue
 
                     data += fh.read()
-                    if 'ProtectedData' in data:
+                    if b'ProtectedData' in data:
                         break   # found DRM voucher
             else:
                 raise Exception("The .kfx-zip archive contains an encrypted DRMION file without a DRM voucher")
@@ -72,7 +72,7 @@ class KFXZipBook:
                 continue
 
             try:
-                voucher = ion.DrmIonVoucher(BytesIO(data), pid[:dsn_len], pid[dsn_len:])
+                voucher = DrmIonVoucher(BytesIO(data), pid[:dsn_len], pid[dsn_len:])
                 voucher.parse()
                 voucher.decryptvoucher()
                 break

DeDRM_plugin/kindlekey.py

@@ -28,6 +28,7 @@ __version__ = '3.0'
 #  2.5   - Final Fix for Windows user names with non-ascii characters, thanks to oneofusoneofus
 #  2.6   - Start adding support for Kindle 1.25+ .kinf2018 file
 #  2.7   - Finish .kinf2018 support, PC & Mac by Apprentice Sakuya
+#  2.8   - Fix for Mac OS X Big Sur
 #  3.0   - Python 3 for calibre 5.0
 
 
@@ -1174,8 +1175,11 @@ elif isosx:
 
         libcrypto = find_library('crypto')
         if libcrypto is None:
-            raise DrmException("libcrypto not found")
+            libcrypto = '/usr/lib/libcrypto.dylib'
-        libcrypto = CDLL(libcrypto)
+        try:
+            libcrypto = CDLL(libcrypto)
+        except Exception as e:
+            raise DrmException("libcrypto not found: " % e)
 
         # From OpenSSL's crypto aes header
         #

DeDRM_plugin/openssl_des.py

@@ -76,7 +76,7 @@ def load_libcrypto():
             return ob.raw
         def decrypt(self, data):
             if not data:
-                return ''
+                return b''
             i = 0
             result = []
             while i < len(data):
@@ -84,6 +84,6 @@ def load_libcrypto():
                 processed_block = self.desdecrypt(block)
                 result.append(processed_block)
                 i += 8
-            return ''.join(result)
+            return b''.join(result)
 
     return DES

FAQs.md

@@ -18,7 +18,7 @@ Just download and use these tools, that's all! Uh, almost. There are a few, uh,
 
 But otherwise, if your ebook is from Amazon, Kobo, Barnes & Noble or any of the ebook stores selling ebooks compatible with Adobe Digital Editions 2.0.1, you should be able to remove the DRM that's been applied to your ebooks.
 
-### A Recent Change to Kindle for PC/Kindle for Mac
+### Recent Changes to Kindle for PC/Kindle for Mac
 Starting with version 1.19, Kindle for PC/Mac uses Amazon's new KFX format which isn't quite as good a source for conversion to ePub as the older KF8 (& MOBI) formats. There are two options to get the older formats. Either stick with version 1.17 or earlier, or modify the executable by changing a file name (PC) or disabling a component of the application (Mac). 
 
 Version 1.17 of Kindle is no longer available directly from Amazon, so you will need to search for the proper file name and find it on a third party site. The name is `KindleForPC-installer-1.17.44170.exe` for PC and `KindleForMac-44182.dmg` for Mac.
@@ -34,7 +34,7 @@ Verify the one of the following cryptographic hash values, using software of you
 * SHA-1: 7AB9A86B954CB23D622BD79E3257F8E2182D791C
 * SHA-256: 28DC21246A9C7CDEDD2D6F0F4082E6BF7EF9DB9CE9D485548E 8A9E1D19EAE2AC. 
 
-You will need to go to the preferences and uncheck the auto update checkbox. Then download and install 1.17 over the top of the newer installation. You'll also need to delete the KFX folders from your My Kindle Content folder.
+You will need to go to the preferences and uncheck the auto update checkbox. Then download and install 1.17 over the top of the newer installation. You'll also need to delete the KFX folders from your My Kindle Content folder. You may also need to take further action to prevent an auto update. The simplest wayis to find the 'updates' folder and replace it with a file. See [this thread] (http://www.mobileread.com/forums/showthread.php?t=283371) at MobileRead for a Script to do this on a PC. On a Mac you can find the folder at ~/Library/Application Support/Kindle/ just delete the folder 'updates' and save a blank text file called 'updates' in its place.
 
 Another possible solution is to use 1.19 or later, but disable KFX by renaming or disabling a necessary component of the application. This may or may not work on versions after 1.25. In a command window, enter the following commands when Kindle for PC/Mac is not running:
 
@@ -51,7 +51,7 @@ Mac Note: If the chmod command fails with a permission error try again using `su
 After restarting the Kindle program any books previously downloaded in KFX format will no longer open. You will need to remove them from your device and re-download them. All future downloads will use the older Kindle formats instead of KFX although they will continue to be placed in one individual subdirectory per book. Note that books soudl be downoad by right-click and 'Download', not by just opening the book. Recent (1.25+) versions of Kindle for Mac/PC may convert KF8 files to a new format that is not supported by these tools when the book is opened for reading.
 
 #### Decrypting KFX
-Thanks to work by several people, the tools can now decrypt KFX format ebooks from Kindle for Mac/PC. In addition to the DeDRM plugin, calibre users will also need to install jhowell's KFX Input plugin which is available through the standard plugin menu in calibre, or directly from [his plugin thread](https://www.mobileread.com/forums/showthread.php?t=291290) on Mobileread. 
+Thanks to work by several people, the tools can now decrypt KFX format ebooks from Kindle for Mac/PC version 1.26 or earlier (version later than 1.26 use a new encryption scheme for KFX files). In addition to the DeDRM plugin, calibre users will also need to install jhowell's KFX Input plugin which is available through the standard plugin menu in calibre, or directly from [his plugin thread](https://www.mobileread.com/forums/showthread.php?t=291290) on Mobileread. 
 
 #### Thanks
 Thanks to jhowell for his investigations into KFX format and the KFX Input plugin. Some of these instructions are from [his thread on the subject](https://www.mobileread.com/forums/showthread.php?t=283371) at MobileRead.
@@ -68,7 +68,7 @@ Install calibre. Install the DeDRM\_plugin in calibre. Install the Obok\_plugin
 # Installing the Tools
 ## The calibre plugin
 ### I am trying to install the calibre plugin, but calibre says "ERROR: Unhandled exception: InvalidPlugin: The plugin in u’[path]DeDRM\_tools\_6.5.3.zip’ is invalid. It does not contain a top-level \_\_init\_\_.py file"
-You are trying to add the tools archive (e.g. `DeDRM_tools_6.5.3.zip`) instead of the plugin. The tools archive is not the plugin. It is a collection of DRM removal tools which includes the plugin. You must unzip the archive, and install the calibre plugin `DeDRM_plugin.zip` from a folder called `DeDRM_calibre_plugin` in the unzipped archive.
+You are trying to add the tools archive (e.g. `DeDRM_tools_6.8.0.zip`) instead of the plugin. The tools archive is not the plugin. It is a collection of DRM removal tools which includes the plugin. You must unzip the archive, and install the calibre plugin `DeDRM_plugin.zip` from a folder called `DeDRM_calibre_plugin` in the unzipped archive.
 
 ### I’ve unzipped the tools archive, but I can’t find the calibre plugin when I try to add them to calibre. I use Windows.
 You should select the zip file that is in the `DeDRM_calibre_plugin` folder, not any files inside the plugin’s zip archive. Make sure you are selecting from the folder that you created when you unzipped the tools archive and not selecting a file inside the still-zipped tools archive.

README.md

@@ -1,24 +1,27 @@
-# DeDRM_tools
-DeDRM tools for ebooks
 # [Guide] How to remove DRM
 Refer to [Wiki Page](https://github.com/apprenticeharper/DeDRM_tools/wiki/Exactly-how-to-remove-DRM)
-# calibre 5.0
 
-Yes, we know that the plugin doesn't work with calibre 5.0. It will eventually, but there's a lot of code to be translated that does complicated stuff with bits and bytes that must be converted by hand. It'll take a while.
+# DeDRM_tools
+DeDRM tools for ebooks
 
+This is a repository that tracks all the scripts and other tools for removing DRM from ebooks that I could find, committed in date order as best as I could manage. (Except for the Requiem tools for Apple's iBooks, and Convert LIT for Microsoft's .lit ebooks.) This includes the tools from a time before Apprentice Alf had a blog, and continues through to when Apprentice Harper (with help) took over maintenance of the tools.
 
-This is a repository of all the scripts and other tools for removing DRM from ebooks that I could find, committed in date order as best as I could manage. (Except for the Requiem tools for Apple's iBooks, and Convert LIT for Microsoft's .lit ebooks.)
+The individual scripts are now released as two plugins for calibre: DeDRM and Obok. 
+The DeDRM plugin handles books that use Amazon DRM, Adobe Digital Editions DRM (version 1), Barnes & Noble DRM, and some historical formats.
+The Obok plugin handles Kobo DRM.
 
-Mostly it tracks the tools released by Apprentice Alf, athough it also includes the individual tools and their histories from before Alf had a blog.
+Users with calibre 5.x or later should use release 7.0.0b2 or later of the tools.
+Users with calibe 4.x or earlier should use release 6.8.0 of the tools.
 
-Users should download the latest zip archive.
 Developers might be interested in forking the repository, as it contains unzipped versions of those tools that are zipped to make the changes over time easier to follow.
 
 For the latest Amazon KFX format, users of the calibre plugin should also install the KFX Input plugin from the standard calibre plugin menu. It's also available from the MobileRead thread here: https://www.mobileread.com/forums/showthread.php?t=291290
 
+Note that DRM can only be removed from KFX format files downloaded with Kindle for PC/Mac 1.26 or earlier. Amazon changes the DRM for KFX files in Kindle for PC/Mac 1.27 and later.
+
 I welcome contributions from others to improve these tools, from expanding the range of books handled, improving key retrieval,  to just general bug fixes, speed improvements and UI enhancements.
 
-I urge people to read the FAQs. But to cover the most common: Use ADE 2.0.1 to be sure not to get the new DRM scheme that these tools can't handle. Do remember to unzip the downloaded archive to get the plugin. You can't load the whole archive into calibre.
+I urge people to read the FAQs. But to cover the most common: Use ADE 2.0.1 to be sure not to get the new DRM scheme that these tools can't handle. Do remember to unzip the downloaded archive to get the plugin (beta versions may be just the plugin  don't unzip that). You can't load the whole tools archive into calibre.
 
 My special thanks to all those developers who have done the hard work of reverse engineering to provide the initial tools.
 

ReadMe_Overview.txt

@@ -26,6 +26,8 @@ The tools are provided in the form of plugins for calibre. Calibre is an open so
 
 DeDRM plugin for calibre (Mac OS X, Windows)
 -------------------------------------------------------
+calibe 5.x and later are now written in Python 3, and plugins must also use Python 3. If you have calibre 5, you must use version 7.x or later of the plugins. For calibre 4.x and earlier, use version 6.8.0 of the plugins.
+
 The DeDRM plugin for calibre removes DRM from your Kindle and Adobe DRM ebooks when they are imported to calibre. Just install the DeDRM plugin (DeDRM_plugin.zip), following the instructions and configuration directions provided in the ReadMe file and the help links in the plugin's configuration dialogs.
 
 Once installed and configured, you can simply add a DRM book to calibre and a DRM-free version will be imported into the calibre database. Note that DRM removal only occurs on IMPORT not on CONVERSION or at any other time. If you have already imported DRMed books you'll need to remove the books from calibre and re-import them.