Query Fields
Query fields are GraphQL fields. scrapqd.gql has all the graphql related implementation. Refer GraphQL documentation for more information.
Query type, Leaves type and group are categorized in scrapqd based on their role. But for graphql all the fields are created in same manner.
How to create query fields
Query fields are creating using GraphQLField
attaching to resolver function. Resolver function will be invoked by graphql to process the query.
Resolver function
- resolver(parser: Parser, info: ResolveInfo, xpath, **kwargs)
Resolver function for the graphql field.
- parser
Parser instance passed down from parent query.
- info
GraphQLResolveInfo instance which gives resolver information.
- xpath
path to locate node(tag).
- kwargs
any additional parameters defined in the GraphQL field.
GraphQL Field
- class Field
GraphQL field class is used to create scrapqd query
- type
GraphQL field type. Mostly
GenericScalar
type is used for fields in the scrapqd library.
- args
Dictionary of arguments for the field in query ex: xpath, name. This should be argumented in resolver function above.
- resolve
resolver function which will be invoked while querying. Above resolver function should be given here.
- description
This description will be shown in the graphql ui for documentation.
Example: Text field
Resolver function
@with_error_traceback
def resolve_text(parser: Parser, info: ResolveInfo,
xpath, data_type=const.DATA_TYPE_DEFAULT_VALUE, multi=const.MULTI_DEFAULT_VALUE):
"""Extracts node(tag) content using given XPath.
:param parser: Parser instance passed down from parent query.
:param info: GraphQLResolveInfo instance which gives resolver information.
:param xpath: path to locate node(tag).
:param data_type: Extracted text will be always in text format. When the data type is provided,
content is converted to that format and returned to the client.
Accepted data types:
- text (default)
- int
- float
:param multi: by default, it is set to False. Thus, when the given xpath locates multiple nodes,
it returns first node value. if it is set `true`, it will return all the node values" \
as list.Given data type is applied to all the nodes individually.
:return:
- text - when multi is set to False, This option can be overridden to return list with single value using `NON_MULTI_RESULT_LIST`.
- List - when multi is set to True
"""
key = get_key(info)
parser.datatype_check(key, data_type)
result = parser.extract_text(key=key, multi=multi, xpath=xpath)
result = parser.data_conversion(result, data_type)
result = parser.get_multi_results(multi, result)
parser.caching(key, result)
return result
Query Field
text = Field(GenericScalar,
args={
'xpath': Argument(NonNull(String), description=const.xpath_desc),
'data_type': Argument(DataTypeEnum, description="data type which should be converted"),
'multi': Argument(Boolean, description=const.multi_desc),
},
resolve=resolve_text,
description="Extracts text content from the give xpath")