acservice package documentation

Base class

class acservice.base.BaseACService

Bases: object

Base class for Audio Commons Service. An Audio Commons service should be a composition of BaseACService and a number of mixins from the classes below (those supported by the service api).

API_BASE_URL = 'http://example.com/api/'
NAME = 'Service name'
URL = 'http://example.com'
add_response_warning(msg)
clear_response_warnings()
collect_response_warnings()
components
configure(config)
get_service_description()

Returns a structured description of the capabilities of each component implemented by the service. Uses each component’s ‘component_description’ method. :return: dict with components as keys

id
implemented_components = None
name
send_request(url, method='get', params=None, data=None, supported_auth_methods=None, account=None, use_authentication_method=None)

Make a request to the service. If not provided, this method automatically chooses a suitable authentication method for making the request. :param method: request method (either ‘get’ or ‘post’) :param url: endpoint api url :param params: request parameters in a dictionary :param data: dictionary of data to be included as json body :param supported_auth_methods: auth methods supported by the api endpoint (defaults to those defined for the service) :param account: user account (for enduser authentication only) :return: dictionary of json response (can raise exception if status_code!=200)

service_id = None
set_service_id(service_id)

This should be a unique id for the service. The id is provided by the Audio Commons consortium. :param service_id: 8 character alphanumeric string (e.g. ef21b9ad)

url
validate_response_status_code(response)

Process service API responses and raise exceptions if errors occur. Otherwise return response as dictionary object loaded from json contents. This base class contains a basic implementation of this method that raises generic exceptions without explanation or details. Services will want to override this method to better interpret the way errors are returned (and provide clearer response to users). :param response: response object (of type requests.models.Response) :return: dictionary including json contents of the response

Authentication mixin

class acservice.auth.ACServiceAuthMixin

Bases: object

Mixin that stores service credentials and implements service linking steps. This mixin implements standard linking strategy for services that support Oauth2 authentication. Services with specific requirements should override the methods from this mixin.

ACCESS_TOKEN_URL = 'http://example.com/api/oauth2/access_token/'
BASE_AUTHORIZE_URL = 'http://example.com/api/authorize/?client_id={0}'
REFRESH_TOKEN_URL = 'http://example.com/api/oauth2/refresh_token/'
SUPPORTED_AUTH_METHODS = ['apikey_auth', 'enduser_auth']
access_token_request_data(authorization_code=None, refresh_token=None)
check_credentials_are_valid(credentials)

Check if the provided credentials are valid for a given service. This method should be overwritten by each individual service or it will always return True. :param credentials: credentials object as stored in ServiceCredentials entry

check_credentials_should_be_renewed_background(credentials)

Check if the provided credentials for a given service should be renewed for new ones. For OAuth2 based services, this should be True when refresh token is about to expire but can be still set to False if only access token has expired (as the access token can be automatically renewed at request time if the refresh token is still valid). This method should be overwritten by each individual service or it will return the opposite value of check_credentials_are_valid. :param credentials: credentials object as stored in ServiceCredentials entry

conf_auth(config)
get_access_token_from_credentials(credentials)

Return the access token from service credentials stored in ServiceCredentials object. This method should be overwritten by each individual service that uses access tokens. :param credentials: credentials object as stored in ServiceCredentials entry :return: access token extracted from the stored credentials

get_apikey()

API key used for non-end user authenticated requests TODO: this should include the way in which the api key is included (via header, request param, etc) :return: string containing the api key

get_auth_info_for_request(auth_method, account=None)

Return dictionary with information about how to authenticate a request. The dictionary can contain the following fields:

  • header: dictionary with header name and header contents (key, value) to be added to a request (including credentials).
  • params: dictionary with request paramer name and contents (key, value) to be added to a request (including credentials).
An example for ‘header’:
{‘headers’: {‘Authorization’: ‘Token API_KEY’}’}
An example for ‘param’:
{‘params’: {‘token’: ‘API_KEY’}}

If both fields are included, both will be added when sending the request. :param auth_method: auth method for which information is wanted :param account: user account (for enduser authentication only) :return: dictionary with auth information

static get_authorize_popup_specs()
get_authorize_url()
get_enduser_token(account)

Get token used to make requests to the service on behalf of ‘account’ TODO: this should include the way in which the token is included (via header, request param, etc) :param account: user account to act on behalf of :return: string containing the token

get_redirect_uri()
get_refresh_token_from_credentials(credentials)

Return the refresh token from service credentials stored in ServiceCredentials object. This method should be overwritten by each individual service that uses access tokens. :param credentials: credentials object as stored in ServiceCredentials entry :return: refresh token extracted from the stored credentials

static process_credentials(credentials_data)
renew_access_token(refresh_token)
renew_credentials(credentials)
request_access_token(authorization_code)
request_credentials(authorization_code)
service_client_id = None
service_client_secret = None
set_credentials(client_id, client_secret)
supports_auth(auth_type)

Licensing mixin

class acservice.licensing.ACLicensingMixin

Bases: object

Mixin that defines methods to allow licensing of Audio Commons content. Services are expected to override methods to adapt them to their own APIs.

LICENSING_ACID_DOMAINS = []
conf_licensing(conf)
describe_licensing()

Returns structured representation of component capabilities Component capabilities include a list of acid_domains which indicate for which domain of resources the service provides licensing for (i.e., ‘Jamendo’ domain means all resources identified by Jamendo:xxx) :return: tuple with (component name, dictionary with component capabilities)

get_licensing_url(context, acid, *args, **kwargs)

Given an Audio Commons unique resource identifier (acid), this function returns a url where the resource can be licensed. If the 3rd party service can’t license that resource or some other errors occur during the collection of the url, an AC exceptions should be raised. Individual services can extend this method with extra parameters to make it more suitable to their needs (e.g., to call the method given an already retrieved resource and avoid in this way an extra request). :param context: Dict with context information for the request (see api.views.get_request_context) :param acid: Audio Commons unique resource identifier :return: url to license the input resource (string)

license(context, acid, *args, **kwargs)

This endpoint returns a license url along with a list of warnings that might contain relevant information for the application. To get the URL, it uses ‘get_licensing_url’ method, therefore ‘get_licensing_url’ is the main method that should be overwritten by third party services. Raise warnings using the BaseACService.add_response_warning method. :param context: Dict with context information for the request (see api.views.get_request_context) :param acid: Audio Commons unique resource identifier :return: url where to get a license

Search mixins

class acservice.search.ACServiceTextSearchMixin

Bases: acservice.search.BaseACServiceSearchMixin

Mixin that defines methods to allow text search. Services are expected to override methods to adapt them to their own APIs.

TEXT_SEARCH_ENDPOINT_URL = 'http://example.com/api/search/'
build_filter_string(filter_input_value)

This method gets a filter string defined using the Audio Commons filter string syntax and Audio Commons field names and values, and returns a filter string which uses the filtering syntax, filter names and compatible values of the individual third party service. Raises ACFilterParsingException if problems occur during filter parsing. For instance, an input filter like “ac:format:wav AND ac:duration:[2,10]” could be translated to something like “format=wav+duration=[2 TO 10]”. :param filter_input_value: input filter string :return: output (translated) filter string

conf_textsearch(*args)

Add SEARCH_TEXT_COMPONENT to the list of implemented components. Also search for methods in the class that have been annotated with the property ‘_translates_filter_for_field_name’. These will be methods decorated with the ‘translates_filter_for_field’ decorator. Then register methods in self.translate_filter_methods_registry so that these can be accessed later.

describe_textsearch()

Returns structured representation of component capabilities :return: tuple with (component name, dictionary with component capabilities)

direct_filters_mapping

Return a dictionary of Audio Commons filter names that can be directly mapped to service resource filters. ‘directly mapped’ means that the value for a given filter can be passed as specified using the Audio Commons filter syntax can be directly used to specify the same filter for the third party service, with the only difference of (probably) changing the field name. For example, if a third party seconds uses the time unit seconds to filter by duration, then there is no need to transform that value when interpreting an Audio Commons filter and passing it to the third party service. In this case the filter can be defined in the direct_filters_mapping function by adding an entry in the dictionary of the Audio Commons field name and the corresponding third party service field name. This works similar to BaseACServiceSearchMixin.direct_fields_mapping property. :return: dictionary mapping (keys are Audio Commons field names and values are services’ resource field names)

get_supported_filters()

Checks which AudioCommons filters can be translated to the third party service filters. These are the filters defined with the decorator @translates_filter_for_field :return: list of available AudioCommons field names (fields equivalent to the filters)

get_supported_sorting_criteria()

Checks which AudioCommons sorting criteria are supported by the third party service. These are the fields that raise an exception when calling ‘process_s_query_parameter’ with ‘raise_exception_if_unsupported’ set to True. :return: list of available AudioCommons sorting criteria

process_f_query_parameter(f)

Process contents of query filter and translate it to corresponding query parameter(s) for the third party service. Raise warnings using the BaseACService.add_response_warning method. The query parameters are returned as a dictionary where keys and values will be sent as keys and values of query parameters in the request to the third party service. Typically the returned query parameters dictionary will only contain one key/value pair. :param f: query filter :return: query parameters dict

process_filter_element(elm, filter_list)

In the Audio Commons API filters are passed as a string which can represent complex structures. For instance a filter could be defined as “ac:format:wav AND ac:duration:[10,40]”. ACServiceTextSearchMixin parses this string (see services.acservice.utils.parse_filter) and transforms it into a nested list of elements. This method takes one of this elements and processes it accordingly. To “process” a filter element means to first identify what kind of element it is. Elements can be:

    1. a filter term like (“field_name”, ”:” “filter_value”)
    1. an operator like (“AND”)
    1. a more complex structure like (X, Y, Z) or (X, (Y, Z)

Filter elements which are not “final ” (complex structures) are processed recursively element by element. Filter elements which are final (filter terms or operators) are rendered calling the ACServiceTextSearchMixin.render_filter_term and ACServiceTextSearchMixin.render_operator_term methods, which are supposed to be overwritten by services which support filtering queries. Before rendering the filters, this method calls ACServiceTextSearchMixin.translate_filter to get the translated field names and values that are understood by the third party service. :param elm: filter element as returned by parser (will be of type pyparsing.ParseResults) :param filter_list: list of processed filter element that’s recursively passed to process_filter_element :return: None (output must be read from filter_list, see ACServiceTextSearchMixin.build_filter_string)

process_q_query_parameter(q)

Process contents of textual query input parameter and translate it to corresponding query parameter(s) for the third party service. Raise warnings using the BaseACService.add_response_warning method. The query parameters are returned as a dictionary where keys and values will be sent as keys and values of query parameters in the request to the third party service. Typically the returned query parameters dictionary will only contain one key/value pair. :param q: textual input query :return: query parameters dict

process_s_query_parameter(s, desc, raise_exception_if_unsupported=False)

Process contents of sort parameter and translate it to corresponding query parameter(s) for the third party service. Raise warnings using the BaseACService.add_response_warning method. The query parameters are returned as a dictionary where keys and values will be sent as keys and values of query parameters in the request to the third party service. Typically the returned query parameters dictionary will only contain one key/value pair. :param s: sorting method :param desc: use descending order :param raise_exception_if_unsupported: whether to raise an exception if desired criteria is unsupported :return: query parameters dict

render_filter_term(key, value_text=None, value_number=None, value_range=None)

This function gets a filter key (field name) and value pair (a filter term) and renders it as a string representing that filter term according to the third party service filtering syntax. The value can be a string, a number or a range, and will be passed to value_text, value_number or value_range accordingly. For example, for a given input key=”format” and value_text=”wav”, this function might return the string “key:wav” if the third party service uses a syntax like “key:value” for filtering. Similarly, for a given input like key=”duration” and value_range=(4,6), the output could be “duration:[4 TO 6]”. Range values do not necessarily need to be of numbers, these could also be text. :param key: filter key (field name) :param value_text: value for the filter in case it is of type string (can be None) :param value_number: value for the filter in case it is of type int of float (can be None) :param value_range: value for the filter in case it is of type tuple (can be None). :return: string with the rendered filter term.

render_operator_term(operator)

Similarly to ACServiceTextSearchMixin.render_filter_term, this method takes as input an string representing an operator in the Audio Commons filter syntax domain, and returns the string corresponding to the same operator in the third party service domain. Audio Commons operators can be “AND”, “OR” or “NOT”. For example, for a given input operator=”AND” this method might return ” && ” if filter terms are “ANDed” using spaces and the characters && in the third party service filter syntax. :param operator: string which can be “AND”, “OR” or “NOT”. :return: string with the corresponding operator in the third party service filter syntax

This function a search request to the third party service and returns a formatted json response as a dictionary if the response status code is 200 or raises an exception otherwise.

Note that to implement text search services do not typically need to overwrite this method but the individual process_x_query_parameter methods.

During processing of the response a number of warnings can be raised using the BaseACService.add_response_warning method. This warnings should contain additional relevant information regarding the request/response that will be returned in the aggregated response. For example, if a request wants to retrieve a number of metadata fields and one of these fields is not supported by the third party service, this will be recorded as a warning. We want to return the other supported fields but also a note that says that field X was not returned because it is not supported by the service.

Common search parameters include: TODO: write common params when decided

Parameters:
  • context – Dict with context information for the request (see api.views.get_request_context)
  • q – textual input query
  • f – query filter
  • s – sorting criteria
  • common_search_params – dictionary with other search parameters commons to all kinds of search
Returns:

formatted text search response as dictionary

translate_filter(ac_field_name, value)

Given an Audio Commons field name and a value for a filter (e.g. “ac:duration” and “3.5”), this method returns the corresponding field name and values that can be understood by a third party service. Following from the previous example, this method could return something like (“duration”, 3500) if the field for “ac:duration” is named “duration” in the third party service and durations are specified in milliseconds instead of seconds. To perform this translation first we check if the field is available in self.direct_filters_mapping. If that is the case, this function simply returns the corresponding (key, value) tuple with the key being changed to the one specified in self.direct_filters_mapping. If field name is not available in self.direct_filters_mapping, then we check if it is available in the registry of translate filters for field methods self.translate_filter_methods_registry which is built when running self.conf_textsearch() (see ACServiceTextSearchMixin.conf_textsearch(self, *args). If a method for the ac_field_name exists in self.translate_filter_methods_registry we call it and return its response. If field does not exist in self.direct_filters_mapping or self.translate_filter_methods_registry we raise an exception to inform that filter for field could not be translated. :param ac_field_name: Audio Commons name of the field to filter :param value: value for the filter :return: (field_name, value) tuple with field_name and value translated to the third party service domain

translate_filter_methods_registry = None
class acservice.search.BaseACServiceSearchMixin

Bases: object

Base class for search-related mixins. This class is in charge of providing necessary methods for handling translation of metadata field names and values between the 3rd party service and the Audio Commons API and ecosystem. In this way, when 3rd party service returns a list of results with services particular fields and values, we can translate these to a unified Audio Commons format. Services that implement any of the search functionalities must at least implement:

  • BaseACServiceSearchMixin.format_search_response(self)
  • BaseACServiceSearchMixin.direct_fields_mapping(self) and/or necessary methods for translating individual fields using the ‘translates_field’ decorator
SERVICE_ID_FIELDNAME = 'id'
add_extra_search_query_params()

Return a dictionary with any extra query parameters in key/value pairs that should be added to the search request. :return: query parameters dict

Search for methods in the class that have been annotated with the property ‘_translates_field_name’. These will be methods decorated with the ‘translates_field’ decorator. Then register methods in self.translate_field_methods_registry so that these can be accessed later.

direct_fields_mapping

Return a dictionary of Audio Commons field names that can be directly mapped to service resource fields. ‘directly mapped’ means that the value can be passed to the response unchanged and only the field name needs (probably) to be changed. For example, id service provider returns a result such as {‘original_filename’: ‘audio filename’}, we just need to change ‘original_filename’ for ‘name’ in order to make it compatible with Audio Commons format. Therefore, the direct_fields_mapping dictionary would include an entry like ‘AUDIO_COMMONS_TERM_FOR_FIELD_NAME’: ‘original_filename’. :return: dictionary mapping (keys are Audio Commons field names and values are services’ resource field names)

format_search_response(response, common_search_params, format)

Take the search request response returned from the service and transform it to the unified Audio Commons search response definition.

Parameters:
  • response – dictionary with json search response
  • common_search_params – common search parameters passed here in case these are needed somewhere
  • format – format with which the response should be returned. Defaults to JSON.
Returns:

dictionary with search results properly formatted

get_num_results_from_response(response)

Given the complete response of a search request to the end service, return the total number of results. :param response: dictionary with the full json response of the request :return: number of total results (integer)

get_results_list_from_response(response)

Given the complete response of a search request to the end service, return the list of results. :param response: dictionary with the full json response of the request :return: list of dict where each dict is a single result

get_supported_fields()

Checks which AudioCommons fields can be translated to the third party service fields. These are the fields supported in ‘direct_fields_mapping’ and those added in the ‘translate_field_methods_registry’ using the @translates_field decorator. :return: list of available AudioCommons field names

id_prefix
process_common_search_params(common_search_params)

This method calls all the functions that process common search parameters (i.e. process_x_query_parameter) and aggregates their returned query parameters for the third party service request. Raise warnings using the BaseACService.add_response_warning method. :param common_search_params: common search query parameters as parsed in the corresponding API view :return: query parameters dict

process_page_query_parameter(page, common_search_params)

Process ‘page’ search query parameter and translate it to corresponding query parameter(s) for the third party service. Raise warnings using the BaseACService.add_response_warning method. The query parameters are returned as a dictionary where keys and values will be sent as keys and values of query parameters in the request to the third party service. Typically the returned query parameters dictionary will only contain one key/value pair. :param page: requested page number (int) :param common_search_params: dictionary with other common search query parameters (might not be needed) :return: query parameters dict

process_size_query_parameter(size, common_search_params)

Process ‘size’ search query parameter and translate it to corresponding query parameter(s) for the third party service. Raise warnings using the BaseACService.add_response_warning method. The query parameters are returned as a dictionary where keys and values will be sent as keys and values of query parameters in the request to the third party service. Typically the returned query parameters dictionary will only contain one key/value pair. :param size: number of desired results per page (int) :param common_search_params: dictionary with other common search query parameters (might not be needed) :return: query parameters dict

translate_field(ac_field_name, result)

Given an audio commons field name and a dictionary representing a single result entry form a service response, return the corresponding field value compatible with the Audio Commons API. To perform this translation first we check if the field is available in self.direct_fields_mapping. If that is the case, this function simply returns the corresponding value according to the field name mapping specified in self.direct_fields_mapping. If field name is not available in self.direct_fields_mapping, then we check if it is available in the registry of translate field methods self.translate_field_methods_registry which is built when running self.conf_search() (see BaseACServiceSearchMixin.conf_search(self, *args). If a method for the ac_field_name exists in self.translate_field_methods_registry we call it and return its response. If field does not exist in self.direct_fields_mapping or self.translate_field_methods_registry we raise an exception to inform that field could not be translated. :param ac_field_name: name of the field in the Audio Commons API domain :param result: dictionary representing a single result entry form a service response :return: value of ac_field_name for the given result

translate_field_id(result)

Default implementation for the translation of ID field. It takes the id of the resource coming from the service and appends the service name. The id of the resource is taken using the SERVICE_ID_FIELDNAME which defaults to ‘id’. If this is not the way in which id is provided by the service then either SERVICE_ID_FIELDNAME is assigned a different value or this function must be overwritten. :param result: dictionary representing a single result entry form a service response :return: id to uniquely identify resource within the Audio Commons Ecosystem

translate_field_methods_registry = None
translate_single_result(result, target_fields, format)

Take an individual search result from a service response in the form of a dictionary and translate its keys and values to an Audio Commons API compatible format. This method iterates over a given set of target ac_fields and computes the value that each field should have in the Audio Commons API context. :param result: dictionary representing a single result entry form a service response :param target_fields: list of Audio Commons fields to return :param format: format with which the response should be returned. Defaults to JSON. :return: dictionary representing the single result with keys and values compatible with Audio Commons API

This decorator annotates the decorated function with a ‘_translates_field_name’ property with a reference to an Audio Commons metadata field name. The ‘_translates_field_name’ property is used when BaseACServiceSearchMixin objects are initialized to build a registry of methods that can “translate” a specific Audio Commons field (see BaseACServiceSearchMixin.conf_search() :param field_name: Audio Commons metadata field name (see services.mixins.constants.py) :return: decorated function

This decorator annotates the decorated function with a ‘_translates_filter_for_field_name’ property with a reference to an Audio Commons metadata field name. The ‘_translates_filter_for_field_name’ property is used when ACServiceTextSearchMixin objects are initialized to build a registry of methods that can “translate” a filter for a specific Audio Commons field (see ACServiceTextSearchMixin.conf_textsearch()). :param field_name: Audio Commons metadata field name (see services.mixins.constants.py) :return: decorated function

Util functions

Parse a filter string using the ‘filterExpr’ grammar defined above. This function returns a pyparsing.ParseResults object that should be further processed with the build_filter_string method of a ACServiceTextSearchMixin instance.

Return CC license name from license URL