Streaming Stripe Data: Easy Guide

Is streaming Stripe Data something you want to learn how to do? If your answer is yes, then you are on just the right page. In this blog, you will learn about all things Stripe, what it is, how it works and most importantly how to work with streaming Stripe data.

Here’s a snapshot of everything you will be looking at:

• What is Stripe?
• How Stripe Works
• Extract Data from Stripe
• Stripe API to a Data Warehouse – Streaming Stripe Data
• Conclusion

What is Stripe?

Stripe is a US-based Financial Service & Software as a Service (SaaS) company. Stripe offers payment processing tools/software along with APIs for any type of businesses that use e-commerce or other types of businesses that need payment gateway whether they are operating web apps or mobile apps. Stripe may as well be called the “Full Stack Payment Processor” or a Payment Gateway. It means that Stripe allows the business to process credit and debit cards, as well as automated clearing house (ACH) transactions including both offline and online transactions. 

How Stripe Works

Stripe provides an extensive methodology in which many complex processes are running. In simple words, it provides the flexibility to its users to transfer money from a customer’s bank account into your business account by a way of both credit & debit card transactions. 

Stripe is really special. To process online transactions you need two things:- 

• A Payment Gateway 
• A Payment Processor 

The purpose of a gateway is to securely capture and transmit the customer’s credit card payment information to the payment processor.

The Payment Processor then processes the transaction in a way that funds from the customer’s bank are then temporarily routed to a merchant account. The payment is then routed to the merchant’s bank account after the credit card-related fees are deducted from the sum from the merchant’s account. 

Stripe does an amazing job in combining payment gateway and payment processing functionality and makes it convenient & may be the cheapest to handle eCommerce. 

Now let’s take a look at how Stripe makes it happen. 

Extract Data from Stripe

Stripe is an API based product. It provides a set of APIs & tools that enables businesses to manage & accept online payments. It is a web API following the RESTful principles, they try to use as many as possible HTTP built-in features to make it accessible to off-the-shelf HTTP clients and they respond back in JSON format.

They also have two types of keys that are used for authentication:

• Testing Mode
• Live Mode

The purpose of the Testing Mode is to do all the testing & play around without messing around with real-time data or system. 

Also, the call needs to be made in HTTPS and not in HTTP as it won’t get accepted. Moreover, it’s also good for security purposes. 

Currently, the stripe API is built around the following ten core resources:

1. Balance – This object is used to show the current balance at your end.
2. File uploads – There may be various occasions when you want to upload files in Stripe like when uploading a receipt.
3. Charges – To charge you the amount as per the order through debit or credit card.
4. Dispute – A dispute may occur when a customer questions your payment with their bank. 
5. Refunds – Refund objects let you make a refund of a charge that has previously been created but not yet refunded.
6. Events – Events are the way to let clients know if something interesting is there for them to see. 
7. Transfers – When you initiate a transfer to a bank account or when Stripe sends you money.  
8. Transfer reversals – A transfer created previously can be reversed if it has not been paid out at that time.
9. Tokens – Tokens can be created with a publishable API key.
10. Customers – Customer objects allow us to track multiple transactions with the same customers by assigning them unique ids.

All the above-mentioned resources support CRUD operations by using HTTP verbs on their associated endpoints. Also as a web API, it can be accessed by using tools like CURL or Postman or any other technology stack i.e. language or framework. 

Some of the options available are:-

• Hyper for Rust
• Python HTTP-client
• Spray-client for Scala
• Ruby Rest Client
• Apache HTTP Client for Java

Let’s assume we want to analyse the payment details of customers for a Superstore in a specific region. For this we will be needing customers data living in that area. To do that we’ll have to call the customer objects that Stripe gives access to for our company. We can do that with the following command:

curl https://api.stripe.com/v1/charges?limit=2 
   -u sk_test_VBokiqTYvBiI2HlSgE4jht2:

and the typical response looks like this:- 

{
  "object": "list",
  "url": "/v1/charges",
  "has_more": false,
  "data": [
    {
      "id": "ch_17SY5f2eZvKYlo2CiUibn76Q",
      "object": "charge",
      "amount": 1000,
      "amount_refunded": 0,
      "application_fee": null,
      "balance_transaction": "txn_67GHbfrewu78ikjQ",
      "captured": true,
      "created": 1476837901,
      "currency": "usd",
      "customer": null,
      "description": "test@hevodata.com Account Credit",
      "destination": null,
      "dispute": null,
      "failure_code": null,
      "failure_message": null,
      "fraud_details": {
      }, …….

In the customer object a list of subscription objects is present that looks like the following JSON document:

{
  "id": "sub_7hy2ghTYujhgdwU87Q",
  "object": "subscription",
  "application_fee_percent": 5,
  "cancel_at_period_end": false,
  "canceled_at": xxxx,
  "current_period_end": 134567897,
  "current_period_start": 11334598,
  "customer": "cus_7hy0yQ34rhbcsWE",
  "discount": xxxxx,
  "ended_at": xxxxx,
  "metadata": {
  },
  "plan": {
    "id": "silver67845",
    "object": "plan",
    "amount": 3000,
    "created": 345678123,
    "currency": "dollar",
    "interval": "quarter-year",
    "interval_count": 1,
    "livemode": false,
    "metadata": {
    },
    "name": "Silver ",
    "statement_descriptor": null,
    "trial_period_days": null
  },
  "quantity": 3,
  "start": 154628765,
  "status": "active",
  "tax_percent": null,
  "trial_end": null,
  "trial_start": null
}

These objects together with part of the customer object contain all the information needed to analyse customers data. This information needs to be extracted, then mapped to the schema of the data warehouse repository & then loaded into it by following the instructions of this post. 

Stripe API to a Data Warehouse – Streaming Stripe Data

Streaming data infrastructure can be set up that will collect Stripe data and push it into a data warehouse in a streaming fashion. Webhooks play an important role in providing the functionality that Stripe supports. You have to register some events to it and every time there is any trigger, Stripe will push a message to your webhook.

For further detail, please check the official API documentation on webhooks. 

Conclusion

Overall, Stripe API is very powerful and helps in extracting data in stream & batch formats as required by clients. It is organized around REST hence providing a robust methodology for its usage. However, setting up the streaming API manually can be difficult and time-consuming. There is a simpler way of streaming Stripe data.

Visit our Website to Explore Hevo

Hevo is a No-code Data Pipeline. It supports pre-built integrations from 100+ data sources at a reasonable price. Hevo will completely automate your data flow from your source to destination and provide a simplified analysis of it.

Explore more about Hevo and Sign Up for a 14-day free trial today. You can now transfer data from sources like Stripe to your target destination for Free using Hevo!

Have any further queries on streaming Stripe data? Let us know in the comments section below.