Package net.sansa_stack.spark.io.rdf.loader

Class AsyncRdfParserHadoop

java.lang.Object

net.sansa_stack.spark.io.rdf.loader.AsyncRdfParserHadoop

public class AsyncRdfParserHadoop
extends Object

Async parsing RDF on a single node using hadoop

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	AsyncRdfParserHadoop.Builder<T>

Constructor Summary

Constructors

Constructor	Description
AsyncRdfParserHadoop()

Method Summary

Modifier and Type	Method	Description
static void	parse(org.apache.hadoop.fs.Path file, org.apache.jena.riot.RDFFormat rdfFormat, org.apache.hadoop.conf.Configuration conf, org.apache.jena.riot.system.StreamRDF sink)
static <T> void	parseRaw(org.apache.hadoop.fs.Path inputFile, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.mapreduce.InputFormat<?,T> inputFormat, ExecutorService executorService, org.apache.jena.riot.system.StreamRDF sink, BiConsumer<T,org.apache.jena.riot.system.StreamRDF> sendRecordToStreamRDF)	The sink must be started beforehand!

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

AsyncRdfParserHadoop

public AsyncRdfParserHadoop()

Method Details

parse

public static void parse(org.apache.hadoop.fs.Path file,
 org.apache.jena.riot.RDFFormat rdfFormat,
 org.apache.hadoop.conf.Configuration conf,
 org.apache.jena.riot.system.StreamRDF sink)
                  throws Exception

Parameters:

file -

rdfFormat - The language of the file. If it is null then probing will be performed.

conf -

sink -

Throws:

Exception

parseRaw

public static <T> void parseRaw(org.apache.hadoop.fs.Path inputFile,
 org.apache.hadoop.conf.Configuration conf,
 org.apache.hadoop.mapreduce.InputFormat<?,T> inputFormat,
 ExecutorService executorService,
 org.apache.jena.riot.system.StreamRDF sink,
 BiConsumer<T,org.apache.jena.riot.system.StreamRDF> sendRecordToStreamRDF)
                         throws IOException,
InterruptedException,
ExecutionException

The sink must be started beforehand!

Type Parameters:

T -

Parameters:

inputFile -

conf -

inputFormat -

executorService - The Executorservice must be closed externally.

sink -

sendRecordToStreamRDF -

Throws:

IOException

InterruptedException

ExecutionException