Metadata-Version: 1.1
Name: bitmath
Version: 1.0.2
Summary: Pythonic module for representing and manipulating file sizes with different prefix notations.
Home-page: https://github.com/tbielawa/bitmath
Author: Tim Bielawa
Author-email: timbielawa@gmail.com
License: MIT
Description: **NOTE: If you're viewing this page on PyPi I strongly urge you to read it on GitHub instead. Over there the Markdown formatting is rendered much more pleasantly.**
        
        - https://github.com/tbielawa/bitmath
        
        bitmath
        =======
        
        bitmath simplifies many facets of interacting with file sizes in
        various units. Examples include: converting between SI and NIST prefix
        units (GiB to kB), converting between units of the same type (SI to
        SI, or NIST to NIST), basic arithmetic operations (subtracting 42KiB
        from 50GiB), and rich comparison operations (1024 Bytes == 1KiB).
        
        In addition to the conversion and math operations, bitmath provides
        human readable representations of values which are suitable for use in
        interactive shells as well as larger scripts and applications.
        
        In discussion we will refer to the NIST units primarily. I.e., instead
        of "megabyte" we will refer to "mibibyte". The former is 10^3 =
        1,000,000 bytes, whereas the second is 2^20 = 1,048,576 bytes. When
        you see file sizes in your file browser, or transfer rates in your web
        browser, what you're really seeing are the base-2 sizes/rates.
        
        
        Basics
        ======
        
        **Class Initializer Signature**
        
            BitMathType([value=0, [bytes=None, [bits=None]]])
        
        A bitmath type may be initialized in four different ways:
        
        - Set no initial value
        
        The default size is 0
        
            zero_kib = KiB()
        
        - Set the value **in current prefix units**
        
        That is to say, if you want to encapsulate 1KiB, initialize the
        bitmath type with ``1``:
        
            one_kib = KiB(1)
        
        	one_kib = KiB(value=1)
        
        - Set the number of bytes
        
        Use the ``bytes`` keyword
        
            one_kib = KiB(bytes=1024)
        
        - Set the number of bits
        
        Use the ``bits`` keyword
        
            one_kib = KiB(bits=8192)
        
        
        Available Classes
        -----------------
        
        There are two **fundamental** classes available:
        
        - ``Bit(Byte)``
        - ``Byte(object)``
        
        There are 24 other classes available, representing all the prefix
        units from "k" through "e" (kilo/kibi through exa/exbi).
        
        Classes with 'i' in their names are **NIST** type classes. They were
        defined by the National Institute of Standards and Technolog (NIST) as
        the 'Binary Prefix Units'. They are defined by increasing powers of 2.
        
        Classes without the 'i' character are **SI** type classes. Though not
        formally defined by any standards organization, they follow the
        International System of Units (SI) pattern (commonly used to
        abbreviate base 10 values). You may hear these referred to as the
        "Decimal" or "SI" prefixes.
        
        - ``Eb(Bit)``
        - ``EB(Byte)``
        - ``Eib(Bit)``
        - ``EiB(Byte)``
        - ``Gb(Bit)``
        - ``GB(Byte)``
        - ``Gib(Bit)``
        - ``GiB(Byte)``
        - ``kb(Bit)``
        - ``kB(Byte)``
        - ``Kib(Bit)``
        - ``KiB(Byte)``
        - ``Mb(Bit)``
        - ``MB(Byte)``
        - ``Mib(Bit)``
        - ``MiB(Byte)``
        - ``Pb(Bit)``
        - ``PB(Byte)``
        - ``Pib(Bit)``
        - ``PiB(Byte)``
        - ``Tb(Bit)``
        - ``TB(Byte)``
        - ``Tib(Bit)``
        - ``TiB(Byte)``
        
        **Note**: Yes, as per SI definition, the ``kB`` class begins with a lower-case 'k' character.
        
        The majority of the functionality of bitmath object comes from their
        rich implementation of standard Python operations. You can use bitmath
        objects in **almost all** of the places you would normally use an
        integer or a float. See **Usage** below for more details.
        
        
        Instance Methods
        ----------------
        
        bitmath objects come with one basic method: ``to_THING()``.
        
        Where ``THING`` is any of the bitmath types. You can even
        ``to_THING()`` an instance into itself again:
        
            In [1]: from bitmath import *
        
            In [2]: one_mib = MiB(1)
        
            In [3]: one_mib_in_kb = one_mib.to_kb()
        
            In [4]: one_mib == one_mib_in_kb
        
            Out[4]: True
        
            In [5]: another_mib = one_mib.to_MiB()
        
            In [6]: print one_mib, one_mib_in_kb, another_mib
        
            1.0MiB 8388.608kb 1.0MiB
        
            In [7]: six_TB = TB(6)
        
            In [8]: six_TB_in_bits = six_TB.to_Bit()
        
            In [9]: print six_TB, six_TB_in_bits
        
            6.0TB 4.8e+13Bit
        
            In [10]: six_TB == six_TB_in_bits
        
            Out[10]: True
        
        
        Instance Attributes
        -------------------
        
        bitmath objects have three public instance attributes:
        
        - ``bytes`` - The number of bytes in the object
        - ``bits`` - The number of bits in the object
        - ``value`` - The value of the instance in **PREFIX** units
        
        For example:
        
            In [13]: dvd_capacity = GB(4.7)
        
            In [14]: print "Capacity in bits: %s\nbytes: %s\n" % \
        
                         (dvd_capacity.bits, dvd_capacity.bytes)
        
               Capacity in bits: 37600000000.0
        
               bytes: 4700000000.0
        
            In [15]: dvd_capacity.value
        
            Out[16]: 4.7
        
        
        Usage
        =====
        
        Supported operations:
        
        - Basic arithmetic: addition, subtraction, multiplication, division
        
        - Size comparison: LT, LE, EQ, NE, GT, GE
        
        - Unit conversion: from bytes through exibytes, supports conversion to any other unit (e.g., Megabytes to Kibibytes)
        
        
        Instantiating any bitmath type is simple:
        
            one_kib = KiB(1)
            KiB(1.0)
        
        Likewise, if you want to represent the same thing in bytes:
        
            one_kib_as_byte = Byte(1024)
            Byte(1024.0)
        
        In these examples ``one_kib`` and ``one_kib_as_byte`` are equivalent
        if tested with the ``==`` operator.
        
        
        See **Examples** for more examples of supported operations.
        
        
        Examples
        ========
        
        Basic unit conversion:
        
            In [1]: from bitmath import *
        
            In [2]: fourty_two_mib = MiB(42)
        
            In [3]: fourty_two_mib_in_kib = fourty_two_mib.to_KiB()
        
            In [4]: fourty_two_mib_in_kib
        
            Out[4]: KiB(43008.0)
        
            In [5]: fourty_two_mib
        
            Out[5]: MiB(42.0)
        
        
        Equality testing:
        
            In [6]: fourty_two_mib == fourty_two_mib_in_kib
        
            Out[6]: True
        
        Basic math:
        
            In [7]: eighty_four_mib = fourty_two_mib + fourty_two_mib_in_kib
        
            In [8]: eighty_four_mib
        
            Out[8]: MiB(84.0)
        
            In [9]: eighty_four_mib == fourty_two_mib * 2
        
            Out[9]: True
        
        
        Real Life Examples
        ==================
        
        **Example 1: Download Speeds**
        
        Let's pretend that your Internet service provider (ISP) advertises
        your maximum downstream as **50Mbps** [1] and you want to know how
        fast that is in mibibytes? ``bitmath`` can do that for you
        easily. Keeping in mind that 1 Byte = 8 bits you can calculate this as
        such:
        
            from bitmath import *
        
            downstream = MB(50)
        
            downstream.to_MiB() / 8
        
            MiB(5.96046447754)
        
        This tells us that if our ISP advertises **50Mbps** we can expect to
        see download rates of nearly **6MiB/sec**.
        
        **[1]** - Assuming your ISP follows the common industry practice of
          using SI (base-10) units to describe file sizes/rates.
        
        **Example 2: Calculating how many files fit on a device**
        
        Given that we have a thumb drive with 12GiB free, how many 4MiB audio
        files can we fit on it?
        
        
            from bitmath import *
        
            thumb_drive = GiB(12)
        
            audio_file = MiB(4)
        
            thumb_drive / audio_file
        
            3072.0
        
        This tells us that we could fit 3072 4MiB audio files on a 12GiB thumb drive.
        
        
        **Example 3: Printing Human-Readable File Sizes in Python**
        
        In a Python script or intrepreter we may wish to print out file sizes
        in something other than bytes (which is what ``os.path.getsize``
        returns). We can use ``bitmath`` to do that too:
        
        
            >>> import os
        
            >>> from bitmath import *
        
            >>> these_files = os.listdir('.')
        
            >>> for f in these_files:
                    f_size = Byte(os.path.getsize(f))
                    print "%s - %s" % (f, f_size.to_KiB())
        
            test_basic_math.py - 3.048828125KiB
            __init__.py - 0.1181640625KiB
            test_representation.py - 0.744140625KiB
            test_to_Type_conversion.py - 2.2119140625KiB
        
        
        **Example 4: Calculating Linux BDP and TCP Window Scaling**
        
        Say we're doing some Linux Kernel TCP performance tuning. For optimum
        speeds we need to calculate our BDP, or Bandwidth Delay Product. For
        this we need to calculate certain values to set some kernel tuning
        parameters to. The point of this tuning is to send the most data we
        can during a measured round-trip-time without sending more than can be
        processed. To accomplish this we are resizing our kernel read/write
        networking/socket buffers.
        
        **Core Networking Values**
        
        - ``net.core.rmem_max`` - **Bytes** - Single Value - Default receive buffer size
        - ``net.core.wmem_max`` - **Bytes** - Single Value - Default write buffer size
        
        **System-Wide Memory Limits**
        
        - ``net.ipv4.tcp_mem`` - **Pages** - Three Value Vector - The ``max`` field of the parameter is the number of **memory pages** allowed for queueing by all TCP sockets.
        
        **Per-Socket Buffers**
        
        Per-socket buffer sizes must not exceede the core networking buffer sizes.
        
        - ``net.ipv4.tcp_rmem`` - **Bytes** - Three Field Vector - The ``max`` field sets the size of the TCP receive buffer
        - ``net.ipv4.tcp_wmem`` - **Bytes** - Three Field Vector - As above, but for the write buffer
        
        We would normally calculate the optimal BDP and related values following this approach:
        
        1. Measure the latency, or round trip time (RTT), between the host we're tuning and our target remote host
        1. Measure/identify our network transfer rate
        1. Calculate the BDP (multiply transfer rate by rtt)
        1. Obtain our current kernel settings
        1. Adjust settings as necessary
        
        But for the sake brevity we'll be working out of an example scenario
        with a pre-defined RTT and transfer rate.
        
        **Scenario**
        
        - We have an average network transfer rate of 1Gb/sec (where ``Gb`` is the SI unit for Gigabits, not Gibibytes)
        - Our latency (RTT) is 0.199ms (milliseconds)
        
        **Calculate Manually**
        
        Lets calculate the BDP now. Because the kernel parameters expect
        values in units of bytes and pages we'll have to convert our transfer
        rate of 1Gb/sec into B/s (Gigabits/second to Bytes/second):
        
        - Convert 1Gb into an equivalent **byte** based unit
        
        Remember 1 Byte = 8 Bytes:
        
            tx_rate_GB = 1/8 = 0.125
        
        Our equivalent transfer rate is 0.125GB/sec.
        
        - Convert our RTT from miliseconds into seconds
        
        Remember 1ms = 10^-3s:
        
            window_seconds = 0.199 * 10^-3 = 0.000199
        
        Our equivalent RTT window is 0.000199s
        
        - Next we multiply the transfer rate by the length of our RTT window (in seconds)
        
        (The unit analysis for this is ``GB/s * s`` leaving us with ``GB``)
        
            BDP = rx_rate_GB * window_seconds = 0.125 * 0.000199 = 0.000024875
        
        Our BDP is 0.000024875GB.
        
        - Convert 0.000024875GB to bytes:
        
        Remember 1GB = 10^9B
        
            BDP_bytes = 0.000024875 * 10^9 = 24875.0
        
        Our BDP is 24875 bytes (or about 24.3KiB)
        
        **Calculate with bitmath**
        
        All of this math can be done much quicker (and with greater accuracy)
        using the bitmath library. Let's see how:
        
            from bitmath import GB
        
            tx = 1/8.0
        
            rtt = 0.199 * 10**-3
        
            bdp = (GB(tx * rtt)).to_Byte()
        
            Byte(24875.0)
        
            bdp.to_KiB()
        
            KiB(24.2919921875)
        
        **Note:** To avoid integer rounding during division, don't forget to divide by ``8.0`` rather than ``8``
        
        We could shorten that even further:
        
            print (GB((1/8.0) * (0.199 * 10**-3))).to_Byte()
        
        	24875.0Byte
        
        **Get the current kernel parameters**
        
        Important to note is that the **per-socket** buffer sizes must not
        exceed the **core network** buffer sizes. Lets fetch our current core
        buffer sizes:
        
            $ sysctl net.core.rmem_max net.core.wmem_max
        
            net.core.rmem_max = 212992
        
            net.core.wmem_max = 212992
        
        Recall, these values are in bytes. What are they in KiB?
        
            Byte(212992).to_KiB()
        
            KiB(208.0)
        
        This means our core networking buffer sizes are set to 208KiB
        each. Now let's check our current per-socket buffer sizes:
        
            $ sysctl net.ipv4.tcp_rmem net.ipv4.tcp_wmem
        
            net.ipv4.tcp_rmem = 4096        87380   6291456
        
            net.ipv4.tcp_wmem = 4096        16384   4194304
        
        Let's double-check that our buffer sizes aren't already out of wack
        (per-socket should be <= networking core)
        
            net_core_max = KiB(bytes=212992)
        
            ipv4_tcp_rmem_max = KiB(bytes=6291456)
        
            ipv4_tcp_rmem_max > net_core_max
        
            True
        
        It appears that my buffers aren't sized appropriately. We'll fix that
        when we set the tunable parameters.
        
        Finally, how large is the entire system TCP buffer?
        
            $ sysctl net.ipv4.tcp_mem
        
            net.ipv4.tcp_mem = 280632       374176  561264
        
        Our max system TCP buffer size is set to **561264**. Recall that this
        parameter is measured in **memory pages**. Most of the time your page
        size is ``4096 bytes``, but you can check by running the command:
        ``getconf PAGESIZE``. To convert the system TCP buffer size
        (561264) into a byte-based unit, we'll multiply it by our pagesize
        (4096):
        
            sys_pages = 561264
        
            page_size = 4096
        
            sys_buffer = Byte(sys_pages * page_size)
        
            print sys_buffer.to_MiB()
        
            2192.4375MiB
        
            print sys_buffer.to_GiB()
        
            2.14105224609GiB
        
        The system max TCP buffer size is about 2.14GiB.
        
        In review, we discovered the following:
        
        * Our **core network** buffer size is insufficient (**212992**), we'll set it higher
        * Our current **per-socket** buffer sizes are **6291456** and **4194304**
        
        And we calculated the following:
        
        * Our ideal **max** per-socket buffer size is **24875** bytes
        * Our ideal **default** per-socket buffer size (half the **max**): **12437**
        
        
        **Finally: Set the new kernel parameters**
        
        Set the **core-network** buffer sizes:
        
            $ sudo sysctl net.core.rmem_max=24875  net.core.wmem_max=24875
        
        	net.core.rmem_max = 4235
        
        	net.core.wmem_max = 4235
        
        Set the **per-socket** buffer sizes:
        
            $ sudo sysctl net.ipv4.tcp_rmem="4096 12437 24875" net.ipv4.tcp_wmem="4096 12437 24875"
        
            net.ipv4.tcp_rmem = 4096 12437 24875
        
            net.ipv4.tcp_wmem = 4096 12437 24875
        
        And it's done! Testing this is left as an exercise for the
        reader. Note that in my experience this is less useful on wireless
        connections.
        
        
        On Units
        ========
        
        As previously stated, in this module you will find two very similar
        sets of classes available. These are the **NIST** and **SI**
        prefixes. The **NIST** prefixes are all base 2 and have an 'i'
        character in the middle. The **SI** prefixes are base 10 and have no
        'i' character.
        
        For smaller values, these two systems of unit prefixes are roughly
        equivalent. The ``round()`` operations below demonstrate how close in
        a percent one "unit" of SI is to one "unit" of NIST.
        
            In [15]: one_kilo = 1 * 10**3
        
            In [16]: one_kibi = 1 * 2**10
        
            In [17]: round(one_kilo / float(one_kibi), 2)
        
            Out[17]: 0.98
        
            In [18]: one_tera = 1 * 10**12
        
            In [19]: one_tebi = 1 * 2**40
        
            In [20]: round(one_tera / float(one_tebi), 2)
        
            Out[20]: 0.91
        
            In [21]: one_exa = 1 * 10**18
        
            In [22]: one_exbi = 1 * 2**60
        
            In [23]: round(one_exa / float(one_exbi), 2)
        
            Out[23]: 0.87
        
        They begin as roughly equivalent, however as you can see, they diverge
        significantly for higher values.
        
        Why two unit systems? Why take the time to point this difference out?
        Why should you care? The Linux Documentation Project comments on
        that:
        
            Before these binary prefixes were introduced, it was fairly common
            to use k=1000 and K=1024, just like b=bit, B=byte.  Unfortunately,
            the M is capital already, and cannot be capitalized to indicate
            binary-ness.
        
            At first that didn't matter too much, since memory modules and
            disks came in sizes that were powers of two, so everyone knew that
            in such contexts "kilobyte" and "megabyte" meant 1024 and 1048576
            bytes, respectively.  What originally was a sloppy use of the
            prefixes "kilo" and "mega" started to become regarded as the "real
            true meaning" when computers were involved.  But then disk
            technology changed, and disk sizes became arbitrary numbers.
            After a period of uncertainty all disk manufacturers settled on
            the standard, namely k=1000, M=1000k, G=1000M.
        
            The situation was messy: in the 14k4 modems, k=1000; in the 1.44MB
            diskettes, M=1024000; etc.  In 1998 the IEC approved the standard
            that defines the binary prefixes given above, enabling people to
            be precise and unambiguous.
        
            Thus, today, MB = 1000000B and MiB = 1048576B.
        
            In the free software world programs are slowly being changed to
            conform.  When the Linux kernel boots and says
        
                hda: 120064896 sectors (61473 MB) w/2048KiB Cache
        
            the MB are megabytes and the KiB are kibibytes.
        
        - Source: ``man 7 units`` - http://man7.org/linux/man-pages/man7/units.7.html
        
        Furthermore, to quote the National Institute of Standards and
        Technology (NIST):
        
            "Once upon a time, computer professionals noticed that 210 was
            very nearly equal to 1000 and started using the SI prefix "kilo"
            to mean 1024. That worked well enough for a decade or two because
            everybody who talked kilobytes knew that the term implied 1024
            bytes. But, almost overnight a much more numerous "everybody"
            bought computers, and the trade computer professionals needed to
            talk to physicists and engineers and even to ordinary people, most
            of whom know that a kilometer is 1000 meters and a kilogram is
            1000 grams.
        
            "Then data storage for gigabytes, and even terabytes, became
            practical, and the storage devices were not constructed on binary
            trees, which meant that, for many practical purposes, binary
            arithmetic was less convenient than decimal arithmetic. The result
            is that today "everybody" does not "know" what a megabyte is. When
            discussing computer memory, most manufacturers use megabyte to
            mean 220 = 1 048 576 bytes, but the manufacturers of computer
            storage devices usually use the term to mean 1 000 000 bytes. Some
            designers of local area networks have used megabit per second to
            mean 1 048 576 bit/s, but all telecommunications engineers use it
            to mean 106 bit/s. And if two definitions of the megabyte are not
            enough, a third megabyte of 1 024 000 bytes is the megabyte used
            to format the familiar 90 mm (3 1/2 inch), "1.44 MB" diskette. The
            confusion is real, as is the potential for incompatibility in
            standards and in implemented systems.
        
            "Faced with this reality, the IEEE Standards Board decided that
            IEEE standards will use the conventional, internationally adopted,
            definitions of the SI prefixes. Mega will mean 1 000 000, except
            that the base-two definition may be used (if such usage is
            explicitly pointed out on a case-by-case basis) until such time
            that prefixes for binary multiples are adopted by an appropriate
            standards body."
        
        Source: http://physics.nist.gov/cuu/Units/binary.html
        
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX
Classifier: Programming Language :: Python
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: System :: Filesystems
Classifier: Topic :: Utilities
