subclassing.py 28 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753
  1. """=============================
  2. Subclassing ndarray in python
  3. =============================
  4. Introduction
  5. ------------
  6. Subclassing ndarray is relatively simple, but it has some complications
  7. compared to other Python objects. On this page we explain the machinery
  8. that allows you to subclass ndarray, and the implications for
  9. implementing a subclass.
  10. ndarrays and object creation
  11. ============================
  12. Subclassing ndarray is complicated by the fact that new instances of
  13. ndarray classes can come about in three different ways. These are:
  14. #. Explicit constructor call - as in ``MySubClass(params)``. This is
  15. the usual route to Python instance creation.
  16. #. View casting - casting an existing ndarray as a given subclass
  17. #. New from template - creating a new instance from a template
  18. instance. Examples include returning slices from a subclassed array,
  19. creating return types from ufuncs, and copying arrays. See
  20. :ref:`new-from-template` for more details
  21. The last two are characteristics of ndarrays - in order to support
  22. things like array slicing. The complications of subclassing ndarray are
  23. due to the mechanisms numpy has to support these latter two routes of
  24. instance creation.
  25. .. _view-casting:
  26. View casting
  27. ------------
  28. *View casting* is the standard ndarray mechanism by which you take an
  29. ndarray of any subclass, and return a view of the array as another
  30. (specified) subclass:
  31. >>> import numpy as np
  32. >>> # create a completely useless ndarray subclass
  33. >>> class C(np.ndarray): pass
  34. >>> # create a standard ndarray
  35. >>> arr = np.zeros((3,))
  36. >>> # take a view of it, as our useless subclass
  37. >>> c_arr = arr.view(C)
  38. >>> type(c_arr)
  39. <class 'C'>
  40. .. _new-from-template:
  41. Creating new from template
  42. --------------------------
  43. New instances of an ndarray subclass can also come about by a very
  44. similar mechanism to :ref:`view-casting`, when numpy finds it needs to
  45. create a new instance from a template instance. The most obvious place
  46. this has to happen is when you are taking slices of subclassed arrays.
  47. For example:
  48. >>> v = c_arr[1:]
  49. >>> type(v) # the view is of type 'C'
  50. <class 'C'>
  51. >>> v is c_arr # but it's a new instance
  52. False
  53. The slice is a *view* onto the original ``c_arr`` data. So, when we
  54. take a view from the ndarray, we return a new ndarray, of the same
  55. class, that points to the data in the original.
  56. There are other points in the use of ndarrays where we need such views,
  57. such as copying arrays (``c_arr.copy()``), creating ufunc output arrays
  58. (see also :ref:`array-wrap`), and reducing methods (like
  59. ``c_arr.mean()``.
  60. Relationship of view casting and new-from-template
  61. --------------------------------------------------
  62. These paths both use the same machinery. We make the distinction here,
  63. because they result in different input to your methods. Specifically,
  64. :ref:`view-casting` means you have created a new instance of your array
  65. type from any potential subclass of ndarray. :ref:`new-from-template`
  66. means you have created a new instance of your class from a pre-existing
  67. instance, allowing you - for example - to copy across attributes that
  68. are particular to your subclass.
  69. Implications for subclassing
  70. ----------------------------
  71. If we subclass ndarray, we need to deal not only with explicit
  72. construction of our array type, but also :ref:`view-casting` or
  73. :ref:`new-from-template`. NumPy has the machinery to do this, and this
  74. machinery that makes subclassing slightly non-standard.
  75. There are two aspects to the machinery that ndarray uses to support
  76. views and new-from-template in subclasses.
  77. The first is the use of the ``ndarray.__new__`` method for the main work
  78. of object initialization, rather then the more usual ``__init__``
  79. method. The second is the use of the ``__array_finalize__`` method to
  80. allow subclasses to clean up after the creation of views and new
  81. instances from templates.
  82. A brief Python primer on ``__new__`` and ``__init__``
  83. =====================================================
  84. ``__new__`` is a standard Python method, and, if present, is called
  85. before ``__init__`` when we create a class instance. See the `python
  86. __new__ documentation
  87. <https://docs.python.org/reference/datamodel.html#object.__new__>`_ for more detail.
  88. For example, consider the following Python code:
  89. .. testcode::
  90. class C(object):
  91. def __new__(cls, *args):
  92. print('Cls in __new__:', cls)
  93. print('Args in __new__:', args)
  94. return object.__new__(cls, *args)
  95. def __init__(self, *args):
  96. print('type(self) in __init__:', type(self))
  97. print('Args in __init__:', args)
  98. meaning that we get:
  99. >>> c = C('hello')
  100. Cls in __new__: <class 'C'>
  101. Args in __new__: ('hello',)
  102. type(self) in __init__: <class 'C'>
  103. Args in __init__: ('hello',)
  104. When we call ``C('hello')``, the ``__new__`` method gets its own class
  105. as first argument, and the passed argument, which is the string
  106. ``'hello'``. After python calls ``__new__``, it usually (see below)
  107. calls our ``__init__`` method, with the output of ``__new__`` as the
  108. first argument (now a class instance), and the passed arguments
  109. following.
  110. As you can see, the object can be initialized in the ``__new__``
  111. method or the ``__init__`` method, or both, and in fact ndarray does
  112. not have an ``__init__`` method, because all the initialization is
  113. done in the ``__new__`` method.
  114. Why use ``__new__`` rather than just the usual ``__init__``? Because
  115. in some cases, as for ndarray, we want to be able to return an object
  116. of some other class. Consider the following:
  117. .. testcode::
  118. class D(C):
  119. def __new__(cls, *args):
  120. print('D cls is:', cls)
  121. print('D args in __new__:', args)
  122. return C.__new__(C, *args)
  123. def __init__(self, *args):
  124. # we never get here
  125. print('In D __init__')
  126. meaning that:
  127. >>> obj = D('hello')
  128. D cls is: <class 'D'>
  129. D args in __new__: ('hello',)
  130. Cls in __new__: <class 'C'>
  131. Args in __new__: ('hello',)
  132. >>> type(obj)
  133. <class 'C'>
  134. The definition of ``C`` is the same as before, but for ``D``, the
  135. ``__new__`` method returns an instance of class ``C`` rather than
  136. ``D``. Note that the ``__init__`` method of ``D`` does not get
  137. called. In general, when the ``__new__`` method returns an object of
  138. class other than the class in which it is defined, the ``__init__``
  139. method of that class is not called.
  140. This is how subclasses of the ndarray class are able to return views
  141. that preserve the class type. When taking a view, the standard
  142. ndarray machinery creates the new ndarray object with something
  143. like::
  144. obj = ndarray.__new__(subtype, shape, ...
  145. where ``subdtype`` is the subclass. Thus the returned view is of the
  146. same class as the subclass, rather than being of class ``ndarray``.
  147. That solves the problem of returning views of the same type, but now
  148. we have a new problem. The machinery of ndarray can set the class
  149. this way, in its standard methods for taking views, but the ndarray
  150. ``__new__`` method knows nothing of what we have done in our own
  151. ``__new__`` method in order to set attributes, and so on. (Aside -
  152. why not call ``obj = subdtype.__new__(...`` then? Because we may not
  153. have a ``__new__`` method with the same call signature).
  154. The role of ``__array_finalize__``
  155. ==================================
  156. ``__array_finalize__`` is the mechanism that numpy provides to allow
  157. subclasses to handle the various ways that new instances get created.
  158. Remember that subclass instances can come about in these three ways:
  159. #. explicit constructor call (``obj = MySubClass(params)``). This will
  160. call the usual sequence of ``MySubClass.__new__`` then (if it exists)
  161. ``MySubClass.__init__``.
  162. #. :ref:`view-casting`
  163. #. :ref:`new-from-template`
  164. Our ``MySubClass.__new__`` method only gets called in the case of the
  165. explicit constructor call, so we can't rely on ``MySubClass.__new__`` or
  166. ``MySubClass.__init__`` to deal with the view casting and
  167. new-from-template. It turns out that ``MySubClass.__array_finalize__``
  168. *does* get called for all three methods of object creation, so this is
  169. where our object creation housekeeping usually goes.
  170. * For the explicit constructor call, our subclass will need to create a
  171. new ndarray instance of its own class. In practice this means that
  172. we, the authors of the code, will need to make a call to
  173. ``ndarray.__new__(MySubClass,...)``, a class-hierarchy prepared call to
  174. ``super(MySubClass, cls).__new__(cls, ...)``, or do view casting of an
  175. existing array (see below)
  176. * For view casting and new-from-template, the equivalent of
  177. ``ndarray.__new__(MySubClass,...`` is called, at the C level.
  178. The arguments that ``__array_finalize__`` receives differ for the three
  179. methods of instance creation above.
  180. The following code allows us to look at the call sequences and arguments:
  181. .. testcode::
  182. import numpy as np
  183. class C(np.ndarray):
  184. def __new__(cls, *args, **kwargs):
  185. print('In __new__ with class %s' % cls)
  186. return super(C, cls).__new__(cls, *args, **kwargs)
  187. def __init__(self, *args, **kwargs):
  188. # in practice you probably will not need or want an __init__
  189. # method for your subclass
  190. print('In __init__ with class %s' % self.__class__)
  191. def __array_finalize__(self, obj):
  192. print('In array_finalize:')
  193. print(' self type is %s' % type(self))
  194. print(' obj type is %s' % type(obj))
  195. Now:
  196. >>> # Explicit constructor
  197. >>> c = C((10,))
  198. In __new__ with class <class 'C'>
  199. In array_finalize:
  200. self type is <class 'C'>
  201. obj type is <type 'NoneType'>
  202. In __init__ with class <class 'C'>
  203. >>> # View casting
  204. >>> a = np.arange(10)
  205. >>> cast_a = a.view(C)
  206. In array_finalize:
  207. self type is <class 'C'>
  208. obj type is <type 'numpy.ndarray'>
  209. >>> # Slicing (example of new-from-template)
  210. >>> cv = c[:1]
  211. In array_finalize:
  212. self type is <class 'C'>
  213. obj type is <class 'C'>
  214. The signature of ``__array_finalize__`` is::
  215. def __array_finalize__(self, obj):
  216. One sees that the ``super`` call, which goes to
  217. ``ndarray.__new__``, passes ``__array_finalize__`` the new object, of our
  218. own class (``self``) as well as the object from which the view has been
  219. taken (``obj``). As you can see from the output above, the ``self`` is
  220. always a newly created instance of our subclass, and the type of ``obj``
  221. differs for the three instance creation methods:
  222. * When called from the explicit constructor, ``obj`` is ``None``
  223. * When called from view casting, ``obj`` can be an instance of any
  224. subclass of ndarray, including our own.
  225. * When called in new-from-template, ``obj`` is another instance of our
  226. own subclass, that we might use to update the new ``self`` instance.
  227. Because ``__array_finalize__`` is the only method that always sees new
  228. instances being created, it is the sensible place to fill in instance
  229. defaults for new object attributes, among other tasks.
  230. This may be clearer with an example.
  231. Simple example - adding an extra attribute to ndarray
  232. -----------------------------------------------------
  233. .. testcode::
  234. import numpy as np
  235. class InfoArray(np.ndarray):
  236. def __new__(subtype, shape, dtype=float, buffer=None, offset=0,
  237. strides=None, order=None, info=None):
  238. # Create the ndarray instance of our type, given the usual
  239. # ndarray input arguments. This will call the standard
  240. # ndarray constructor, but return an object of our type.
  241. # It also triggers a call to InfoArray.__array_finalize__
  242. obj = super(InfoArray, subtype).__new__(subtype, shape, dtype,
  243. buffer, offset, strides,
  244. order)
  245. # set the new 'info' attribute to the value passed
  246. obj.info = info
  247. # Finally, we must return the newly created object:
  248. return obj
  249. def __array_finalize__(self, obj):
  250. # ``self`` is a new object resulting from
  251. # ndarray.__new__(InfoArray, ...), therefore it only has
  252. # attributes that the ndarray.__new__ constructor gave it -
  253. # i.e. those of a standard ndarray.
  254. #
  255. # We could have got to the ndarray.__new__ call in 3 ways:
  256. # From an explicit constructor - e.g. InfoArray():
  257. # obj is None
  258. # (we're in the middle of the InfoArray.__new__
  259. # constructor, and self.info will be set when we return to
  260. # InfoArray.__new__)
  261. if obj is None: return
  262. # From view casting - e.g arr.view(InfoArray):
  263. # obj is arr
  264. # (type(obj) can be InfoArray)
  265. # From new-from-template - e.g infoarr[:3]
  266. # type(obj) is InfoArray
  267. #
  268. # Note that it is here, rather than in the __new__ method,
  269. # that we set the default value for 'info', because this
  270. # method sees all creation of default objects - with the
  271. # InfoArray.__new__ constructor, but also with
  272. # arr.view(InfoArray).
  273. self.info = getattr(obj, 'info', None)
  274. # We do not need to return anything
  275. Using the object looks like this:
  276. >>> obj = InfoArray(shape=(3,)) # explicit constructor
  277. >>> type(obj)
  278. <class 'InfoArray'>
  279. >>> obj.info is None
  280. True
  281. >>> obj = InfoArray(shape=(3,), info='information')
  282. >>> obj.info
  283. 'information'
  284. >>> v = obj[1:] # new-from-template - here - slicing
  285. >>> type(v)
  286. <class 'InfoArray'>
  287. >>> v.info
  288. 'information'
  289. >>> arr = np.arange(10)
  290. >>> cast_arr = arr.view(InfoArray) # view casting
  291. >>> type(cast_arr)
  292. <class 'InfoArray'>
  293. >>> cast_arr.info is None
  294. True
  295. This class isn't very useful, because it has the same constructor as the
  296. bare ndarray object, including passing in buffers and shapes and so on.
  297. We would probably prefer the constructor to be able to take an already
  298. formed ndarray from the usual numpy calls to ``np.array`` and return an
  299. object.
  300. Slightly more realistic example - attribute added to existing array
  301. -------------------------------------------------------------------
  302. Here is a class that takes a standard ndarray that already exists, casts
  303. as our type, and adds an extra attribute.
  304. .. testcode::
  305. import numpy as np
  306. class RealisticInfoArray(np.ndarray):
  307. def __new__(cls, input_array, info=None):
  308. # Input array is an already formed ndarray instance
  309. # We first cast to be our class type
  310. obj = np.asarray(input_array).view(cls)
  311. # add the new attribute to the created instance
  312. obj.info = info
  313. # Finally, we must return the newly created object:
  314. return obj
  315. def __array_finalize__(self, obj):
  316. # see InfoArray.__array_finalize__ for comments
  317. if obj is None: return
  318. self.info = getattr(obj, 'info', None)
  319. So:
  320. >>> arr = np.arange(5)
  321. >>> obj = RealisticInfoArray(arr, info='information')
  322. >>> type(obj)
  323. <class 'RealisticInfoArray'>
  324. >>> obj.info
  325. 'information'
  326. >>> v = obj[1:]
  327. >>> type(v)
  328. <class 'RealisticInfoArray'>
  329. >>> v.info
  330. 'information'
  331. .. _array-ufunc:
  332. ``__array_ufunc__`` for ufuncs
  333. ------------------------------
  334. .. versionadded:: 1.13
  335. A subclass can override what happens when executing numpy ufuncs on it by
  336. overriding the default ``ndarray.__array_ufunc__`` method. This method is
  337. executed *instead* of the ufunc and should return either the result of the
  338. operation, or :obj:`NotImplemented` if the operation requested is not
  339. implemented.
  340. The signature of ``__array_ufunc__`` is::
  341. def __array_ufunc__(ufunc, method, *inputs, **kwargs):
  342. - *ufunc* is the ufunc object that was called.
  343. - *method* is a string indicating how the Ufunc was called, either
  344. ``"__call__"`` to indicate it was called directly, or one of its
  345. :ref:`methods<ufuncs.methods>`: ``"reduce"``, ``"accumulate"``,
  346. ``"reduceat"``, ``"outer"``, or ``"at"``.
  347. - *inputs* is a tuple of the input arguments to the ``ufunc``
  348. - *kwargs* contains any optional or keyword arguments passed to the
  349. function. This includes any ``out`` arguments, which are always
  350. contained in a tuple.
  351. A typical implementation would convert any inputs or outputs that are
  352. instances of one's own class, pass everything on to a superclass using
  353. ``super()``, and finally return the results after possible
  354. back-conversion. An example, taken from the test case
  355. ``test_ufunc_override_with_super`` in ``core/tests/test_umath.py``, is the
  356. following.
  357. .. testcode::
  358. input numpy as np
  359. class A(np.ndarray):
  360. def __array_ufunc__(self, ufunc, method, *inputs, **kwargs):
  361. args = []
  362. in_no = []
  363. for i, input_ in enumerate(inputs):
  364. if isinstance(input_, A):
  365. in_no.append(i)
  366. args.append(input_.view(np.ndarray))
  367. else:
  368. args.append(input_)
  369. outputs = kwargs.pop('out', None)
  370. out_no = []
  371. if outputs:
  372. out_args = []
  373. for j, output in enumerate(outputs):
  374. if isinstance(output, A):
  375. out_no.append(j)
  376. out_args.append(output.view(np.ndarray))
  377. else:
  378. out_args.append(output)
  379. kwargs['out'] = tuple(out_args)
  380. else:
  381. outputs = (None,) * ufunc.nout
  382. info = {}
  383. if in_no:
  384. info['inputs'] = in_no
  385. if out_no:
  386. info['outputs'] = out_no
  387. results = super(A, self).__array_ufunc__(ufunc, method,
  388. *args, **kwargs)
  389. if results is NotImplemented:
  390. return NotImplemented
  391. if method == 'at':
  392. if isinstance(inputs[0], A):
  393. inputs[0].info = info
  394. return
  395. if ufunc.nout == 1:
  396. results = (results,)
  397. results = tuple((np.asarray(result).view(A)
  398. if output is None else output)
  399. for result, output in zip(results, outputs))
  400. if results and isinstance(results[0], A):
  401. results[0].info = info
  402. return results[0] if len(results) == 1 else results
  403. So, this class does not actually do anything interesting: it just
  404. converts any instances of its own to regular ndarray (otherwise, we'd
  405. get infinite recursion!), and adds an ``info`` dictionary that tells
  406. which inputs and outputs it converted. Hence, e.g.,
  407. >>> a = np.arange(5.).view(A)
  408. >>> b = np.sin(a)
  409. >>> b.info
  410. {'inputs': [0]}
  411. >>> b = np.sin(np.arange(5.), out=(a,))
  412. >>> b.info
  413. {'outputs': [0]}
  414. >>> a = np.arange(5.).view(A)
  415. >>> b = np.ones(1).view(A)
  416. >>> c = a + b
  417. >>> c.info
  418. {'inputs': [0, 1]}
  419. >>> a += b
  420. >>> a.info
  421. {'inputs': [0, 1], 'outputs': [0]}
  422. Note that another approach would be to to use ``getattr(ufunc,
  423. methods)(*inputs, **kwargs)`` instead of the ``super`` call. For this example,
  424. the result would be identical, but there is a difference if another operand
  425. also defines ``__array_ufunc__``. E.g., lets assume that we evalulate
  426. ``np.add(a, b)``, where ``b`` is an instance of another class ``B`` that has
  427. an override. If you use ``super`` as in the example,
  428. ``ndarray.__array_ufunc__`` will notice that ``b`` has an override, which
  429. means it cannot evaluate the result itself. Thus, it will return
  430. `NotImplemented` and so will our class ``A``. Then, control will be passed
  431. over to ``b``, which either knows how to deal with us and produces a result,
  432. or does not and returns `NotImplemented`, raising a ``TypeError``.
  433. If instead, we replace our ``super`` call with ``getattr(ufunc, method)``, we
  434. effectively do ``np.add(a.view(np.ndarray), b)``. Again, ``B.__array_ufunc__``
  435. will be called, but now it sees an ``ndarray`` as the other argument. Likely,
  436. it will know how to handle this, and return a new instance of the ``B`` class
  437. to us. Our example class is not set up to handle this, but it might well be
  438. the best approach if, e.g., one were to re-implement ``MaskedArray`` using
  439. ``__array_ufunc__``.
  440. As a final note: if the ``super`` route is suited to a given class, an
  441. advantage of using it is that it helps in constructing class hierarchies.
  442. E.g., suppose that our other class ``B`` also used the ``super`` in its
  443. ``__array_ufunc__`` implementation, and we created a class ``C`` that depended
  444. on both, i.e., ``class C(A, B)`` (with, for simplicity, not another
  445. ``__array_ufunc__`` override). Then any ufunc on an instance of ``C`` would
  446. pass on to ``A.__array_ufunc__``, the ``super`` call in ``A`` would go to
  447. ``B.__array_ufunc__``, and the ``super`` call in ``B`` would go to
  448. ``ndarray.__array_ufunc__``, thus allowing ``A`` and ``B`` to collaborate.
  449. .. _array-wrap:
  450. ``__array_wrap__`` for ufuncs and other functions
  451. -------------------------------------------------
  452. Prior to numpy 1.13, the behaviour of ufuncs could only be tuned using
  453. ``__array_wrap__`` and ``__array_prepare__``. These two allowed one to
  454. change the output type of a ufunc, but, in contrast to
  455. ``__array_ufunc__``, did not allow one to make any changes to the inputs.
  456. It is hoped to eventually deprecate these, but ``__array_wrap__`` is also
  457. used by other numpy functions and methods, such as ``squeeze``, so at the
  458. present time is still needed for full functionality.
  459. Conceptually, ``__array_wrap__`` "wraps up the action" in the sense of
  460. allowing a subclass to set the type of the return value and update
  461. attributes and metadata. Let's show how this works with an example. First
  462. we return to the simpler example subclass, but with a different name and
  463. some print statements:
  464. .. testcode::
  465. import numpy as np
  466. class MySubClass(np.ndarray):
  467. def __new__(cls, input_array, info=None):
  468. obj = np.asarray(input_array).view(cls)
  469. obj.info = info
  470. return obj
  471. def __array_finalize__(self, obj):
  472. print('In __array_finalize__:')
  473. print(' self is %s' % repr(self))
  474. print(' obj is %s' % repr(obj))
  475. if obj is None: return
  476. self.info = getattr(obj, 'info', None)
  477. def __array_wrap__(self, out_arr, context=None):
  478. print('In __array_wrap__:')
  479. print(' self is %s' % repr(self))
  480. print(' arr is %s' % repr(out_arr))
  481. # then just call the parent
  482. return super(MySubClass, self).__array_wrap__(self, out_arr, context)
  483. We run a ufunc on an instance of our new array:
  484. >>> obj = MySubClass(np.arange(5), info='spam')
  485. In __array_finalize__:
  486. self is MySubClass([0, 1, 2, 3, 4])
  487. obj is array([0, 1, 2, 3, 4])
  488. >>> arr2 = np.arange(5)+1
  489. >>> ret = np.add(arr2, obj)
  490. In __array_wrap__:
  491. self is MySubClass([0, 1, 2, 3, 4])
  492. arr is array([1, 3, 5, 7, 9])
  493. In __array_finalize__:
  494. self is MySubClass([1, 3, 5, 7, 9])
  495. obj is MySubClass([0, 1, 2, 3, 4])
  496. >>> ret
  497. MySubClass([1, 3, 5, 7, 9])
  498. >>> ret.info
  499. 'spam'
  500. Note that the ufunc (``np.add``) has called the ``__array_wrap__`` method
  501. with arguments ``self`` as ``obj``, and ``out_arr`` as the (ndarray) result
  502. of the addition. In turn, the default ``__array_wrap__``
  503. (``ndarray.__array_wrap__``) has cast the result to class ``MySubClass``,
  504. and called ``__array_finalize__`` - hence the copying of the ``info``
  505. attribute. This has all happened at the C level.
  506. But, we could do anything we wanted:
  507. .. testcode::
  508. class SillySubClass(np.ndarray):
  509. def __array_wrap__(self, arr, context=None):
  510. return 'I lost your data'
  511. >>> arr1 = np.arange(5)
  512. >>> obj = arr1.view(SillySubClass)
  513. >>> arr2 = np.arange(5)
  514. >>> ret = np.multiply(obj, arr2)
  515. >>> ret
  516. 'I lost your data'
  517. So, by defining a specific ``__array_wrap__`` method for our subclass,
  518. we can tweak the output from ufuncs. The ``__array_wrap__`` method
  519. requires ``self``, then an argument - which is the result of the ufunc -
  520. and an optional parameter *context*. This parameter is returned by
  521. ufuncs as a 3-element tuple: (name of the ufunc, arguments of the ufunc,
  522. domain of the ufunc), but is not set by other numpy functions. Though,
  523. as seen above, it is possible to do otherwise, ``__array_wrap__`` should
  524. return an instance of its containing class. See the masked array
  525. subclass for an implementation.
  526. In addition to ``__array_wrap__``, which is called on the way out of the
  527. ufunc, there is also an ``__array_prepare__`` method which is called on
  528. the way into the ufunc, after the output arrays are created but before any
  529. computation has been performed. The default implementation does nothing
  530. but pass through the array. ``__array_prepare__`` should not attempt to
  531. access the array data or resize the array, it is intended for setting the
  532. output array type, updating attributes and metadata, and performing any
  533. checks based on the input that may be desired before computation begins.
  534. Like ``__array_wrap__``, ``__array_prepare__`` must return an ndarray or
  535. subclass thereof or raise an error.
  536. Extra gotchas - custom ``__del__`` methods and ndarray.base
  537. -----------------------------------------------------------
  538. One of the problems that ndarray solves is keeping track of memory
  539. ownership of ndarrays and their views. Consider the case where we have
  540. created an ndarray, ``arr`` and have taken a slice with ``v = arr[1:]``.
  541. The two objects are looking at the same memory. NumPy keeps track of
  542. where the data came from for a particular array or view, with the
  543. ``base`` attribute:
  544. >>> # A normal ndarray, that owns its own data
  545. >>> arr = np.zeros((4,))
  546. >>> # In this case, base is None
  547. >>> arr.base is None
  548. True
  549. >>> # We take a view
  550. >>> v1 = arr[1:]
  551. >>> # base now points to the array that it derived from
  552. >>> v1.base is arr
  553. True
  554. >>> # Take a view of a view
  555. >>> v2 = v1[1:]
  556. >>> # base points to the view it derived from
  557. >>> v2.base is v1
  558. True
  559. In general, if the array owns its own memory, as for ``arr`` in this
  560. case, then ``arr.base`` will be None - there are some exceptions to this
  561. - see the numpy book for more details.
  562. The ``base`` attribute is useful in being able to tell whether we have
  563. a view or the original array. This in turn can be useful if we need
  564. to know whether or not to do some specific cleanup when the subclassed
  565. array is deleted. For example, we may only want to do the cleanup if
  566. the original array is deleted, but not the views. For an example of
  567. how this can work, have a look at the ``memmap`` class in
  568. ``numpy.core``.
  569. Subclassing and Downstream Compatibility
  570. ----------------------------------------
  571. When sub-classing ``ndarray`` or creating duck-types that mimic the ``ndarray``
  572. interface, it is your responsibility to decide how aligned your APIs will be
  573. with those of numpy. For convenience, many numpy functions that have a corresponding
  574. ``ndarray`` method (e.g., ``sum``, ``mean``, ``take``, ``reshape``) work by checking
  575. if the first argument to a function has a method of the same name. If it exists, the
  576. method is called instead of coercing the arguments to a numpy array.
  577. For example, if you want your sub-class or duck-type to be compatible with
  578. numpy's ``sum`` function, the method signature for this object's ``sum`` method
  579. should be the following:
  580. .. testcode::
  581. def sum(self, axis=None, dtype=None, out=None, keepdims=False):
  582. ...
  583. This is the exact same method signature for ``np.sum``, so now if a user calls
  584. ``np.sum`` on this object, numpy will call the object's own ``sum`` method and
  585. pass in these arguments enumerated above in the signature, and no errors will
  586. be raised because the signatures are completely compatible with each other.
  587. If, however, you decide to deviate from this signature and do something like this:
  588. .. testcode::
  589. def sum(self, axis=None, dtype=None):
  590. ...
  591. This object is no longer compatible with ``np.sum`` because if you call ``np.sum``,
  592. it will pass in unexpected arguments ``out`` and ``keepdims``, causing a TypeError
  593. to be raised.
  594. If you wish to maintain compatibility with numpy and its subsequent versions (which
  595. might add new keyword arguments) but do not want to surface all of numpy's arguments,
  596. your function's signature should accept ``**kwargs``. For example:
  597. .. testcode::
  598. def sum(self, axis=None, dtype=None, **unused_kwargs):
  599. ...
  600. This object is now compatible with ``np.sum`` again because any extraneous arguments
  601. (i.e. keywords that are not ``axis`` or ``dtype``) will be hidden away in the
  602. ``**unused_kwargs`` parameter.
  603. """
  604. from __future__ import division, absolute_import, print_function