Package net.sansa_stack.spark.io.rdf.output

Class RddRdfWriter2

java.lang.Object
net.sansa_stack.spark.io.rdf.output.RddRdfWriter2
public class RddRdfWriter2 extends Object
Core class for configuration and execution of writing RDDs of RDF out using Hadaop. An RDD's number splits into the hadoop conf and the OutputFormat uses it to decide whether to output header / footer data blocks on the first / last partitions.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected boolean
    mapQuadsToTriplesForTripleLangs
     
    protected org.apache.jena.shared.PrefixMapping
    prefixes
     
    protected org.apache.jena.riot.RDFFormat
    rdfFormat
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    RddRdfWriter2(org.apache.jena.riot.RDFFormat rdfFormat, boolean mapQuadsToTriplesForTripleLangs, org.apache.jena.shared.PrefixMapping prefixes)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    protected org.apache.hadoop.conf.Configuration
    buildConfiguration(org.apache.spark.rdd.RDD<?> rdd)
     
    protected void
    configure(org.apache.hadoop.conf.Configuration conf)
     
    static <T> org.apache.spark.api.java.JavaPairRDD<Long,T>
    toPairRdd(org.apache.spark.api.java.JavaRDD<T> rdd)
     
    void
    writeQuads(org.apache.spark.rdd.RDD<org.apache.jena.sparql.core.Quad> rdd, org.apache.hadoop.fs.Path path)
     
    void
    writeTriples(org.apache.spark.rdd.RDD<org.apache.jena.graph.Triple> rdd, org.apache.hadoop.fs.Path path)
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • rdfFormat

      protected org.apache.jena.riot.RDFFormat rdfFormat

    • mapQuadsToTriplesForTripleLangs

      protected boolean mapQuadsToTriplesForTripleLangs

    • prefixes

      protected org.apache.jena.shared.PrefixMapping prefixes

  • Constructor Details

    • RddRdfWriter2

      public RddRdfWriter2(org.apache.jena.riot.RDFFormat rdfFormat, boolean mapQuadsToTriplesForTripleLangs, org.apache.jena.shared.PrefixMapping prefixes)

  • Method Details

    • writeTriples

      public void writeTriples(org.apache.spark.rdd.RDD<org.apache.jena.graph.Triple> rdd, org.apache.hadoop.fs.Path path)

    • writeQuads

      public void writeQuads(org.apache.spark.rdd.RDD<org.apache.jena.sparql.core.Quad> rdd, org.apache.hadoop.fs.Path path)

    • buildConfiguration

      protected org.apache.hadoop.conf.Configuration buildConfiguration(org.apache.spark.rdd.RDD<?> rdd)

    • configure

      protected void configure(org.apache.hadoop.conf.Configuration conf)

    • toPairRdd

      public static <T> org.apache.spark.api.java.JavaPairRDD<Long,T> toPairRdd(org.apache.spark.api.java.JavaRDD<T> rdd)