utils module¶
utils Module¶
Author: | Dominic Hunt |
---|
Functions¶
argProcess (**kwargs) |
|||
callableDetails (item) |
Takes a callable item and extracts the details. | ||
callableDetailsString (item) |
Takes a callable item and returns a string detailing the function. | ||
date () |
Provides a string of today’s date | ||
discountAverage (data, discount) |
An accumulating mean | ||
errorResp () |
Takes an error that has been caught and returns the details as a string | ||
find_class (class_name, class_folder, …[, …]) |
Finds and imports a class from a given folder. | ||
find_function (function_name, function_folder) |
Finds and imports a function from a given folder. | ||
flatten (data) |
Yields the elements in order from any N dimensional iterable | ||
getClassArgs (inspected_class[, arg_ignore]) |
Finds the arguments that could be passed into the specified class | ||
getClassAttributes (inspected_class[, ignore]) |
Finds the public attributes of the specified class | ||
getFuncArgs (inspected_function) |
Finds the arguments that could be passed into the specified function | ||
kendalw (data[, ranked]) |
Calculates Kendall’s W for a n*m array with n items and m ‘judges’. | ||
kendalwt (data[, ranked]) |
Calculates Kendall’s W for a n*m array with n items and m ‘judges’. | ||
kendalwts (data[, ranked]) |
Calculates Kendall’s W for a n*m array with n items and m ‘judges’. | ||
kldivergence (m0, m1, c0, c1) |
Calculates the Kullback–Leibler divergence between two distributions using the means and covariances | ||
listMerge (*args) |
For merging lists with objects that are not solely numbers | ||
listMergeGen (*args) |
Fast merging of lists of numbers | ||
listMergeNP (*args) |
Fast merging of lists of numbers | ||
list_all_equal (data) |
Checks if all of the elements of a list are the same. | ||
mergeDatasetRepr (data[, dataLabel]) |
Take a list of dictionaries and turn it into a dictionary of lists of strings | ||
mergeDatasets (data[, extend]) |
Take a list of dictionaries and turn it into a dictionary of lists of objects | ||
mergeDicts (*args) |
Merges any number of dictionaries with different keys into a new dict | ||
mergeTwoDicts (x, y) |
Given two dicts, merge them into a new dict as a shallow copy | ||
movingaverage (data, windowSize[, edgeCorrection]) |
Average over an array | ||
runningAverage (data) |
An accumulating mean | ||
runningMean (oldMean, newValue, numValues) |
A running mean | ||
runningSTD (oldSTD, oldMean, newMean, newValue) |
|
||
unique (seq[, idfun]) |
Finds the unique items in a list and returns them in order found. | ||
varyingParams (intObjects, params) |
Takes a list of models or tasks and returns a dictionary with only the parameters which vary and their values |
Classes¶
ClassNameError |
|
FunctionNameError |
Class Inheritance Diagram¶
digraph inheritance81d1610b57 { rankdir=LR; size="8.0, 12.0"; "ClassNameError" [URL="#utils.ClassNameError",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top"]; "FunctionNameError" [URL="#utils.FunctionNameError",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top"]; }Author: | Dominic Hunt |
---|
-
utils.
callableDetails
(item)[source]¶ Takes a callable item and extracts the details.
Currently only extracts things stored in
item.Name
anditem.Params
Parameters: item (callable item) – Returns: details – Contains the properties of the Return type: tuple pair with string and dictionary of strings Examples
>>> from utils import callableDetails >>> def foo(): print("foo") >>> foo.Name = "boo" >>> callableDetails(foo) ('boo', None) >>> foo.Params = {1: 2, 2: 3} >>> callableDetails(foo) ('boo', {'1': '2', '2': '3'})
-
utils.
callableDetailsString
(item)[source]¶ Takes a callable item and returns a string detailing the function.
Currently only extracts things stored in
item.Name
anditem.Params
Parameters: item (callable item) – Returns: description – Contains the properties and name of the callable Return type: string Examples
>>> from utils import callableDetailsString >>> def foo(): print("foo") >>> foo.Name = "boo" >>> callableDetailsString(foo) 'boo' >>> foo.Params = {1: 2, 2: 3} >>> callableDetailsString(foo) 'boo with 1 : 2, 2 : 3'
-
utils.
date
()[source]¶ Provides a string of today’s date
Returns: date – The string is of the form [year]-[month]-[day] Return type: string
-
utils.
discountAverage
(data, discount)[source]¶ An accumulating mean
Parameters: Returns: results – The values from the moving average
Return type: ndArray of length data
Examples
>>> discountAverage([1, 2, 3, 4], 1) array([1. , 1.5, 2. , 2.5]) >>> discountAverage([1, 2, 3, 4], 0.25) array([1. , 1.8 , 2.71428571, 3.68235294])
-
utils.
errorResp
()[source]¶ Takes an error that has been caught and returns the details as a string
Returns: description – Contains the description of the error Return type: string
-
utils.
find_class
(class_name, class_folder, inherited_class, excluded_files=None)[source]¶ Finds and imports a class from a given folder. Does not look in subfolders
Parameters: - class_name (string) – The name of the class to be used
- class_folder (basestring) – The path where the class is likely to be found
- inherited_class (class) – The class that the searched for class inherits from
- excluded_files (list, optional) – A list of modules to be excluded from the search. Can be described using portions of file names.
Returns: sought_class – The uninstansiated class sought
Return type: inherited_class
-
utils.
find_function
(function_name, function_folder, excluded_files=None)[source]¶ Finds and imports a function from a given folder. Does not look in subfolders
Parameters: - function_name (string) – The name of the function to be used
- function_folder (basestring) – The path where the function is likely to be found
- excluded_files (list, optional) – A list of modules to be excluded from the search. Can be described using portions of file names.
Returns: sought_class – The uninstansiated class sought
Return type: inherited_class
-
utils.
flatten
(data)[source]¶ Yields the elements in order from any N dimensional iterable
Parameters: data (iterable) – Yields: ID ((string,list)) – A pair containing the value at each location and the co-ordinates used to access them. Examples
>>> a = [[1, 2, 3],[4, 5, 6]] >>> for i, loc in flatten(a): print(i,loc) 1 [0, 0] 2 [0, 1] 3 [0, 2] 4 [1, 0] 5 [1, 1] 6 [1, 2]
-
utils.
getClassArgs
(inspected_class, arg_ignore=[u'self'])[source]¶ Finds the arguments that could be passed into the specified class
-
utils.
getClassAttributes
(inspected_class, ignore=[u'self'])[source]¶ Finds the public attributes of the specified class
-
utils.
getFuncArgs
(inspected_function)[source]¶ Finds the arguments that could be passed into the specified function
Parameters: inspected_function – Returns:
-
utils.
kendalw
(data, ranked=False)[source]¶ Calculates Kendall’s W for a n*m array with n items and m ‘judges’.
Parameters: Returns: w – The Kendall’s W
Return type: Notes
Based on Legendre, P. (2010). Coefficient of Concordance. In Encyclopedia of Research Design (pp. 164–169). 2455 Teller Road, Thousand Oaks California 91320 United States: SAGE Publications, Inc. http://doi.org/10.4135/9781412961288.n55
Examples
>>> data = np.array([[2., 0., 5., 1.], [3., 3., 3., 4.], [1., 5., 3., 5.], [1., 1., 4., 2.], [2., 4., 5., 1.], [1., 0., 0., 2.]])
>>> kendalw(data) 0.22857142857142856
>>> data = np.array([[1, 1, 1, 1], [2, 2, 2, 2], [3, 3, 3, 3], [4, 4, 4, 4], [5, 5, 5, 5], [6, 6, 6, 6]])
>>> kendalw(data) 1.0
-
utils.
kendalwt
(data, ranked=False)[source]¶ Calculates Kendall’s W for a n*m array with n items and m ‘judges’. Corrects for ties.
Parameters: Returns: w – The Kendall’s W
Return type: Notes
Based on Legendre, P. (2010). Coefficient of Concordance. In Encyclopedia of Research Design (pp. 164–169). 2455 Teller Road, Thousand Oaks California 91320 United States: SAGE Publications, Inc. http://doi.org/10.4135/9781412961288.n55
Examples
>>> data = np.array([[2., 0., 5., 1.], [3., 3., 3., 4.], [1., 5., 3., 5.], [1., 1., 4., 2.], [2., 4., 5., 1.], [1., 0., 0., 2.]]) >>> kendalwt(data) 0.24615384615384617
>>> data = np.array([[1, 1, 1, 1], [2, 2, 2, 2], [3, 3, 3, 3], [4, 4, 4, 4], [5, 5, 5, 5], [6, 6, 6, 6]]) >>> kendalwt(data) 1.0
-
utils.
kendalwts
(data, ranked=False)[source]¶ Calculates Kendall’s W for a n*m array with n items and m ‘judges’. Corrects for ties.
Parameters: Returns: w – The Kendall’s W
Return type: Notes
Based on Legendre, P. (2010). Coefficient of Concordance. In Encyclopedia of Research Design (pp. 164–169). 2455 Teller Road, Thousand Oaks California 91320 United States: SAGE Publications, Inc. http://doi.org/10.4135/9781412961288.n55
Examples
>>> data = np.array([[2., 0., 5., 1.], [3., 3., 3., 4.], [1., 5., 3., 5.], [1., 1., 4., 2.], [2., 4., 5., 1.], [1., 0., 0., 2.]]) >>> kendalwts(data) 0.24615384615384617
>>> data = np.array([[1, 1, 1, 1], [2, 2, 2, 2], [3, 3, 3, 3], [4, 4, 4, 4], [5, 5, 5, 5], [6, 6, 6, 6]]) >>> kendalwts(data) 1.0
-
utils.
kldivergence
(m0, m1, c0, c1)[source]¶ Calculates the Kullback–Leibler divergence between two distributions using the means and covariances
Parameters: - m0 (array of N floats) – The means of distribution 0
- m1 (array of N floats) – The means of distribution 1
- c0 (NxN array of floats) – The covariance matrix for distribution 0
- c1 (NxN array of floats) – The covariance matrix for distribution 1
Returns: kl – The Kullback–Leibler divergence
Return type:
-
utils.
listMerge
(*args)[source]¶ For merging lists with objects that are not solely numbers
Parameters: args (list of lists) – A list of 1D lists of objects Returns: combinations – An np.array with len(args) columns and a row for each combination Return type: np.array Examples
>>> listMerge([1, 2, 3], [5, 6, 7]).T array([[1, 2, 3, 1, 2, 3, 1, 2, 3], [5, 5, 5, 6, 6, 6, 7, 7, 7]])
-
utils.
listMergeGen
(*args)[source]¶ Fast merging of lists of numbers
Parameters: args (list of lists of numbers) – A list of 1D lists of numbers Yields: combination (numpy.array of 1 x len(args)) – Array of all combinations Examples
>>> for i in listMergeGen(0.7): print(repr(i)) array([0.7]) >>> for i in listMergeGen([0.7, 0.1]): print(repr(i)) array([0.7]) array([0.1]) >>> for i in listMergeGen([0.7, 0.1], [0.6]): print(repr(i)) array([0.7, 0.6]) array([0.1, 0.6]) >>> for i in listMergeGen([0.7, 0.1], []): print(repr(i))
>>> for i in listMergeGen([0.7, 0.1], 0.6): print(repr(i)) array([0.7, 0.6]) array([0.1, 0.6])
-
utils.
listMergeNP
(*args)[source]¶ Fast merging of lists of numbers
Parameters: args (list of lists of numbers) – A list of 1D lists of numbers Returns: combinations – An np.array with len(args) columns and a row for each combination Return type: np.array Examples
>>> listMergeNP([1, 2, 3], [5, 6, 7]).T array([[1, 2, 3, 1, 2, 3, 1, 2, 3], [5, 5, 5, 6, 6, 6, 7, 7, 7]])
-
utils.
list_all_equal
(data)[source]¶ Checks if all of the elements of a list are the same.
Parameters: data (list of 1D) – The list of elements to compare Returns: equivalence – True if the elements are all the same Return type: bool Notes
-
utils.
mergeDatasetRepr
(data, dataLabel=u'')[source]¶ Take a list of dictionaries and turn it into a dictionary of lists of strings
Parameters: - data (list of dicts containing strings, lists or numbers) –
- dataLabel (string, optional) – This string will be appended to the front of each key in the new dataset Default blank
Returns: newStore – For each key a list will be formed of the string representations of each of the former key values.
Return type: dictionary of lists of strings
-
utils.
mergeDatasets
(data, extend=False)[source]¶ Take a list of dictionaries and turn it into a dictionary of lists of objects
Parameters: - data (list of dicts containing strings, lists or numbers) –
- extend (bool, optional) – If lists should be extended rather than appended. Default False
Returns: newStore – For each key a list will be formed of the former key values. If a data set did not contain a key a value of None will be entered for it.
Return type: dictionary of lists of objects
Examples
>>> data = [{'a':[1, 2, 3], 'b':[7, 8, 9]}, {'b':[4, 5, 6], 'c':'string', 'd':5}] >>> mergeDatasets(data) {'a': [[1, 2, 3], None], 'c': [None, 'string'], 'b': [[7, 8, 9], [4, 5, 6]], 'd': [None, 5]} >>> mergeDatasets(data, extend=True) {'a': [1, 2, 3, None], 'c': [None, 'string'], 'b': [7, 8, 9, 4, 5, 6], 'd': [None, 5]} >>> data = [{'b': np.array([[7, 8, 9], [1, 2, 3]])}, {'b': np.array([[4, 5, 6], [2, 3, 4]])}] >>> mergeDatasets(data, extend = True) {'b': [array([7, 8, 9]), array([1, 2, 3]), array([4, 5, 6]), array([2, 3, 4])]} >>> mergeDatasets(data) {'b': [array([[7, 8, 9], [1, 2, 3]]), array([[4, 5, 6], [2, 3, 4]])]}
-
utils.
mergeDicts
(*args)[source]¶ Merges any number of dictionaries with different keys into a new dict
Precedence goes to key value pairs in latter dicts
Parameters: args (list of dictionaries) – Returns: mergedDict Return type: dictionary
-
utils.
mergeTwoDicts
(x, y)[source]¶ Given two dicts, merge them into a new dict as a shallow copy
Assumes different keys in both dictionaries
Parameters: - x (dictionary) –
- y (dictionary) –
Returns: mergedDict
Return type: dictionary
-
utils.
movingaverage
(data, windowSize, edgeCorrection=False)[source]¶ Average over an array
Parameters: Returns: convolution
Return type: array
Examples
>>> movingaverage([1, 1, 1, 1, 1], 3) array([0.66666667, 1. , 1. , 1. , 0.66666667]) >>> movingaverage([1, 1, 1, 1, 1, 1, 1, 1], 4) array([0.5 , 0.75, 1. , 1. , 1. , 1. , 1. , 0.75]) >>> movingaverage([1, 1, 1, 1, 1], 3, edgeCorrection=True) array([1., 1., 1., 1., 1.])
-
utils.
runningAverage
(data)[source]¶ An accumulating mean
Parameters: data (list or 1-D array of floats) – The set of values to be averaged Returns: results – The values from the moving average Return type: ndArray of length data Examples
>>> runningAverage([1,2,3,4]) array([1. , 1.5, 2. , 2.5])
-
utils.
runningMean
(oldMean, newValue, numValues)[source]¶ A running mean
Parameters: Returns: newMean – The new running average mean
Return type: Notes
Based on Donald Knuth’s Art of Computer Programming, Vol 2, page 232, 3rd edition and taken from https://www.johndcook.com/blog/standard_deviation/
Examples
>>> runningMean(1, 2, 2) 1.5 >>> runningMean(1.5, 3, 3) 2.0
-
utils.
runningSTD
(oldSTD, oldMean, newMean, newValue)[source]¶ Parameters: Returns: newSTD – The new running average standard deviation
Return type: Notes
Based on Donald Knuth’s Art of Computer Programming, Vol 2, page 232, 3rd edition (which is based on B. P. Welford (2012) Note on a Method for Calculating Corrected Sums of Squares and Products, Technometrics, 4:3, 419-420, DOI: 10.1080/00401706.1962.10490022 This version is taken from https://www.johndcook.com/blog/standard_deviation/
Examples
>>> runningSTD(0, 1, 1.5, 2) 0.5
>>> runningSTD(0.5, 1.5, 2.0, 3) 2.0
-
utils.
unique
(seq, idfun=None)[source]¶ Finds the unique items in a list and returns them in order found.
Inspired by discussion on
http://www.peterbe.com/plog/uniqifiers-benchmark
Notably f10 Andrew Dalke and f8 by Dave KirbyParameters: - seq (an iterable object) – The sequence from which the unique list will be compiled
- idfun (function, optional) – A hashing function for transforming the items into the form that is to
be compared. Default is the
None
Returns: result – The list of unique items
Return type: Examples
>>> a=list('ABeeE') >>> unique(a) ['A', 'B', 'e', 'E']
>>> unique(a, lambda x: x.lower()) ['A', 'B', 'e']
Note
Unless order is needed it is best to use list(set(seq))