Doorda Knowledge Base

About Doorda Host

Doorda Host is a highly scalable, enterprise data warehouse designed to give you full access to the Doorda Data Products via a single login. Because there is no infrastructure to manage or data to download, you can focus on analysing data to uncover meaningful insights without the need for a database administrator.

Doorda Host makes it easy to share insights within your organization and beyond using your favourite applications. Doorda Host is built on Google Big Query so it can connect to all the most popular data Visualisation and Analysis tools including Tableau, Power Bi, QLIK, R Studio and Python to name a few.  Any solution with a Big Query Connector can access our Data Products removing the need to learn new applications.

Doorda Host is built on Columnar Database. Why? Standard relational databases are optimised for storing rows of data, a columnar database is designed for fast retrieval of columns of data, typically in analytical applications. Column-oriented tables are a crucial factor in analytic query performance as they drastically reduce the overall disk I/O requirements and minimise the amount of data you need to answer a question.

All our packages include generous querying and data download capabilities based on the following parameters.

Querying

Querying refers to the size of your query and user-defined functions. These apply regardless of how you connect. Doorda Host monitors queries by using one metric: the number of bytes processed.

When you run a query, the size will be dependent on the total data processed in the columns you’ve selected, even if you set an explicit LIMIT on the results. The total bytes per column is calculated based on the size of the column. For example, if you query a column containing 5MB of data, the query size will be 5MB regardless of the result. If you query an entire table of 5GB, e.g. Select *, then the query will be 5GB. You decide how big the query will be, smaller queries are not only more efficient they will also run much faster.

Data Downloads

Data downloads refer to the amount of data downloaded each month. This is based on the results of your query. E.g. if you query a 5GB table for the top 50,000 rows the 50,000 rows returned will be classified as a data download. You are then able to extract the results to your local machine for further analysis or report creation.

You can also connect via Python if you want to create bespoke datasets or automate a feed into your own applications.

Packages

Dependant on the package you’ve signed up for your will be allocated a monthly query and data download limit. Overall use each month is based on the total size of your queries and data downloads as per the above. Your allocation will be reset at the start of each month. The unused data will not be carried over.

E.g. if you have a limit of 500GB of queries and only use 230GB in a month the next month your account will reset to 500GB. The same principle applies to data downloads.

    Attached Files
    There are no attachments for this article.
    Related Articles
    API Documentation
    Viewed 198 times since Wed, Feb 14, 2018
    Connecting to BigQuery with Jetbrains Datagrip (and most IDE)
    Viewed 9 times since Fri, Jun 22, 2018
    Doorda Host Supported Functions and Operators
    Viewed 168 times since Mon, Jul 9, 2018
    Google Cloud Authentication
    Viewed 133 times since Wed, Sep 12, 2018