##############################################################################
#
# Copyright (c) 2005-2007 Zope Corporation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL).  A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################

Transaction support for Blobs
=============================

We need a database with a blob supporting storage::

    >>> from ZODB.MappingStorage import MappingStorage
    >>> from ZODB.Blobs.BlobStorage import BlobStorage
    >>> from ZODB.DB import DB
    >>> import transaction
    >>> import tempfile
    >>> from tempfile import mkdtemp
    >>> base_storage = MappingStorage("test")
    >>> blob_dir = mkdtemp()
    >>> blob_storage = BlobStorage(blob_dir, base_storage)
    >>> database = DB(blob_storage)
    >>> connection1 = database.open()
    >>> root1 = connection1.root()
    >>> from ZODB.Blobs.Blob import Blob

Putting a Blob into a Connection works like any other Persistent object::

    >>> blob1 = Blob()
    >>> blob1.open('w').write('this is blob 1')
    >>> root1['blob1'] = blob1
    >>> transaction.commit()

Aborting a transaction involving a blob write cleans up uncommitted
file data::

    >>> dead_blob = Blob()
    >>> dead_blob.open('w').write('this is a dead blob')
    >>> root1['dead_blob'] = dead_blob
    >>> fname = dead_blob._p_blob_uncommitted
    >>> import os
    >>> os.path.exists(fname)
    True
    >>> transaction.abort()
    >>> os.path.exists(fname)
    False

Opening a blob gives us a filehandle.  Getting data out of the
resulting filehandle is accomplished via the filehandle's read method::

    >>> connection2 = database.open()
    >>> root2 = connection2.root()
    >>> blob1a = root2['blob1']
    >>> blob1a._p_blob_refcounts()
    (0, 0)
    >>>
    >>> blob1afh1 = blob1a.open("r")
    >>> blob1afh1.read()
    'this is blob 1'
    >>> # The filehandle keeps a reference to its blob object
    >>> blob1afh1.blob._p_blob_refcounts()
    (1, 0)

Let's make another filehandle for read only to blob1a, this should bump
up its refcount by one, and each file handle has a reference to the
(same) underlying blob::

    >>> blob1afh2 = blob1a.open("r")
    >>> blob1afh2.blob._p_blob_refcounts()
    (2, 0)
    >>> blob1afh1.blob._p_blob_refcounts()
    (2, 0)
    >>> blob1afh2.blob is blob1afh1.blob
    True

Let's close the first filehandle we got from the blob, this should decrease
its refcount by one::

    >>> blob1afh1.close()
    >>> blob1a._p_blob_refcounts()
    (1, 0)

Let's abort this transaction, and ensure that the filehandles that we
opened are now closed and that the filehandle refcounts on the blob
object are cleared::

    >>> transaction.abort()
    >>> blob1afh1.blob._p_blob_refcounts()
    (0, 0)
    >>> blob1afh2.blob._p_blob_refcounts()
    (0, 0)
    >>> blob1a._p_blob_refcounts()
    (0, 0)
    >>> blob1afh2.read()
    Traceback (most recent call last):
        ...
    ValueError: I/O operation on closed file

If we open a blob for append, its write refcount should be nonzero.
Additionally, writing any number of bytes to the blobfile should
result in the blob being marked "dirty" in the connection (we just
aborted above, so the object should be "clean" when we start)::

    >>> bool(blob1a._p_changed)
    False
    >>> blob1a.open('r').read()
    'this is blob 1'
    >>> blob1afh3 = blob1a.open('a')
    >>> blob1afh3.write('woot!')
    >>> blob1a._p_blob_refcounts()
    (0, 1)
    >>> bool(blob1a._p_changed)
    True

We can open more than one blob object during the course of a single
transaction::

    >>> blob2 = Blob()
    >>> blob2.open('w').write('this is blob 3')
    >>> root2['blob2'] = blob2
    >>> transaction.commit()
    >>> blob2._p_blob_refcounts()
    (0, 0)
    >>> blob1._p_blob_refcounts()
    (0, 0)

Since we committed the current transaction above, the aggregate
changes we've made to blob, blob1a (these refer to the same object) and
blob2 (a different object) should be evident::

    >>> blob1.open('r').read()
    'this is blob 1woot!'
    >>> blob1a.open('r').read()
    'this is blob 1woot!'
    >>> blob2.open('r').read()
    'this is blob 3'

We shouldn't be able to persist a blob filehandle at commit time
(although the exception which is raised when an object cannot be
pickled appears to be particulary unhelpful for casual users at the
moment)::

    >>> root1['wontwork'] = blob1.open('r')
    >>> transaction.commit()
    Traceback (most recent call last):
        ...
    TypeError: coercing to Unicode: need string or buffer, BlobFile found

Abort for good measure::

    >>> transaction.abort()

Attempting to change a blob simultaneously from two different
connections should result in a write conflict error::

    >>> tm1 = transaction.TransactionManager()
    >>> tm2 = transaction.TransactionManager()
    >>> root3 = database.open(transaction_manager=tm1).root()
    >>> root4 = database.open(transaction_manager=tm2).root()
    >>> blob1c3 = root3['blob1']
    >>> blob1c4 = root4['blob1']
    >>> blob1c3fh1 = blob1c3.open('a')
    >>> blob1c4fh1 = blob1c4.open('a')
    >>> blob1c3fh1.write('this is from connection 3')
    >>> blob1c4fh1.write('this is from connection 4')
    >>> tm1.get().commit()
    >>> root3['blob1'].open('r').read()
    'this is blob 1woot!this is from connection 3'
    >>> tm2.get().commit()
    Traceback (most recent call last):
        ...
    ConflictError: database conflict error (oid 0x01, class ZODB.Blobs.Blob.Blob)

After the conflict, the winning transaction's result is visible on both
connections::

    >>> root3['blob1'].open('r').read()
    'this is blob 1woot!this is from connection 3'
    >>> tm2.get().abort()
    >>> root4['blob1'].open('r').read()
    'this is blob 1woot!this is from connection 3'

BlobStorages implementation of getSize() includes the blob data and adds it to
the underlying storages result of getSize(). (We need to ensure the last
number to be an int, otherwise it will be a long on 32-bit platforms and an
int on 64-bit)::

    >>> underlying_size = base_storage.getSize()
    >>> blob_size = blob_storage.getSize()
    >>> int(blob_size - underlying_size)
    91


Savepoints and Blobs
--------------------

We do support optimistic savepoints ::

    >>> connection5 = database.open()
    >>> root5 = connection5.root()
    >>> blob = Blob()
    >>> blob_fh = blob.open("wb")
    >>> blob_fh.write("I'm a happy blob.")
    >>> blob_fh.close()
    >>> root5['blob'] = blob
    >>> transaction.commit()
    >>> root5['blob'].open("rb").read()
    "I'm a happy blob."
    >>> blob_fh = root5['blob'].open("a")
    >>> blob_fh.write(" And I'm singing.")
    >>> blob_fh.close()
    >>> root5['blob'].open("rb").read()
    "I'm a happy blob. And I'm singing."
    >>> savepoint = transaction.savepoint(optimistic=True)
    >>> root5['blob'].open("rb").read()
    "I'm a happy blob. And I'm singing."
    >>> transaction.get().commit()

We do not support non-optimistic savepoints::

    >>> blob_fh = root5['blob'].open("a")
    >>> blob_fh.write(" And the weather is beautiful.")
    >>> blob_fh.close()
    >>> root5['blob'].open("rb").read()
    "I'm a happy blob. And I'm singing. And the weather is beautiful."
    >>> savepoint = transaction.savepoint()             # doctest: +ELLIPSIS
    Traceback (most recent call last):
        ...
    TypeError: ('Savepoints unsupported', <ZODB.Blobs.Blob.BlobDataManager instance at 0x...>)
    >>> transaction.abort()

Reading Blobs outside of a transaction
--------------------------------------

If you want to read from a Blob outside of transaction boundaries (e.g. to
stream a file to the browser), you can use the openDetached() method::

    >>> connection6 = database.open()
    >>> root6 = connection6.root()
    >>> blob = Blob()
    >>> blob_fh = blob.open("wb")
    >>> blob_fh.write("I'm a happy blob.")
    >>> blob_fh.close()
    >>> root6['blob'] = blob
    >>> transaction.commit()
    >>> blob.openDetached().read()
    "I'm a happy blob."

Of course, that doesn't work for empty blobs::

    >>> blob = Blob()
    >>> blob.openDetached()
    Traceback (most recent call last):
        ...
    BlobError: Blob does not exist.

nor when the Blob is already opened for writing::

    >>> blob = Blob()
    >>> blob_fh = blob.open("wb")
    >>> blob.openDetached()
    Traceback (most recent call last):
        ...
    BlobError: Already opened for writing.

You can also pass a factory to the openDetached method that will be used to
instantiate the file. This is used for e.g. creating filestream iterators::

    >>> class customfile(file):
    ...   pass
    >>> blob_fh.write('Something')
    >>> blob_fh.close()
    >>> fh = blob.openDetached(customfile)
    >>> fh  # doctest: +ELLIPSIS
    <open file '...', mode 'rb' at 0x...>
    >>> isinstance(fh, customfile)
    True


Note: Nasty people could use a factory that opens the file for writing. This
would be evil.

It does work when the transaction was aborted, though::

    >>> blob = Blob()
    >>> blob_fh = blob.open("wb")
    >>> blob_fh.write("I'm a happy blob.")
    >>> blob_fh.close()
    >>> root6['blob'] = blob
    >>> transaction.commit()

    >>> blob_fh = blob.open("wb")
    >>> blob_fh.write("And I'm singing.")
    >>> blob_fh.close()
    >>> transaction.abort()
    >>> blob.openDetached().read()
    "I'm a happy blob."


Teardown
--------

We don't need the storage directory and databases anymore::

    >>> import shutil
    >>> shutil.rmtree(blob_dir)
    >>> tm1.get().abort()
    >>> tm2.get().abort()
    >>> database.close()
