## The Problem

Recently, at the company where I work, we faced the challenge of building a marketplace solution that could scale and support vast traffic. It’s for a niche market, but the specifications and requirements are quite similar to already available solutions like Vinted, eBay, or Depop. These platforms have established reputations, and the goal for the company was to bring all sellers in their industry together to trade in one place.

## The Goal

For the Minimum Viable Product (MVP), there was quite an extensive list of features and capabilities expected, including:

- Onboarding customers and sellers with full payment provider setup
- Product listings
- Product approval flow via the super admin
- Fast search and product discovery
- Cart-to-checkout flow
- Buy now option
- Offers/product negotiation flow
- Messaging between seller and buyer
- Order management and shipping
- Return flow
- Notifications throughout the system
- Opt-in or opt-out of certain features like offers and returns

## Selecting the Tech Stack

As this blog focuses primarily on backend technology, I will keep the discussion limited to backend stacks.

Based on project needs and our proficiency in Node.js (Typescript), we selected it for backend API development. The main question was whether to write the API from scratch or extend an existing solution.

Weighing the two options:

- Writing the system from scratch (using NestJS) provides full control, allowing us to add features like messaging directly into the system while avoiding unnecessary bloat.
- Using an existing solution, like Medusa JS, a highly extensible e-commerce platform, allows us to leverage a battle-tested e-commerce flow and only extend it for custom logic and additional marketplace-specific features. However, this comes with the tradeoff of unused features that are not needed by the marketplace.

## Decision to Use Medusa

In the end, we decided to pick Medusa for the following reasons:

- Written in Typescript
- Uses ORMs we're familiar with (TypeORM)
- Highly extensible and customizable
- Well-documented
- Modular architecture in V2 (coming soon), which helps address the issue of unused features/modules
- Great developer adoption and support

The rest of the tech stack was decided based on Medusa, as it integrates seamlessly with many existing adapters.

## Selecting the Database – PostgreSQL

PostgreSQL is the primary supported database for Medusa. We initially tried CockroachDB but encountered issues with certain data types, so we opted for the recommended PostgreSQL, which we set up using Google Cloud SQL.

## Implementing Authentication

Medusa comes with built-in authentication, but we needed social authentication (at least Google). Fortunately, there is a [Medusa plugin](https://medusajs.com/plugins/medusa-plugin-auth/) for this, which made it easy to set up. We just needed to configure Google credentials and environment variables.

## Writing `medusa-plugin-marketplace`

This being the third marketplace I’ve built with Medusa, I’ve come across things I wished were essentially plug-and-play to transform a fresh Medusa installation into a basic marketplace. So, I decided to consolidate these elements and publish them as a public plugin: [`medusa-plugin-marketplace`](https://www.npmjs.com/package/medusa-plugin-marketplace).

It includes the following features:

- Multiple store support in the Medusa store model
- Store-specific scoping for:
    - Products
    - Orders
    - Shipping options
    - Invites
    - Users
- Store roles and permissions
- Middleware to prevent cross-store actions
- Advanced Stripe module with marketplace logic:
    - Application fee support
    - Payment splitting (e.g., shipping fee goes to the marketplace Stripe account if it handles label printing)
    - Payment split handling on returns
    - Customizable payment intent and capture behavior
    - Stripe account onboarding widget on super admin
- Order cart splitting for multiple stores
- Support for filtering public products by store ID

The plugin is still a work-in-progress, and more work is needed to make it customizable to user needs. Please [give it a star](https://github.com/Slaknoah/medusa-marketplace) to support the work.

## Cache Service – Redis

Medusa supports both in-memory and Redis caching via plugins. For production use, Redis is recommended, so we opted for Redis.

## Event Listeners and Storage

We implemented the Redis event module for Medusa, which is essential for listening to certain events and triggering actions, such as:

- Deleting products or variants when the quantity is updated
- Sending notifications (more on that later)
- Removing products from the search index upon deletion
- Syncing product categories upon option updates
- Updating offers when orders are completed, etc.

## Fast Search/Product Indexing with Algolia

We implemented Algolia search but found that the existing module wasn’t customizable enough for our business needs. Since our products are volatile and constantly deleted once out of stock, I wrote a custom Algolia search module for Medusa. This reduced the payload saved in the index. Although this approach comes at a cost with increased search traffic, it allowed us to get the MVP to market quickly. However, we are considering switching to Meilisearch in the future due to the costs associated with Algolia.

## Choosing Google Cloud for Hosting and Solving Issues with Storage

We opted for Google Cloud App Engine to deploy the backend because of its scalability and integration with other dependencies like the database and storage. Though documentation for deploying Medusa to App Engine is scarce, setting it up was straightforward with the right IAM configurations and a continuous deployment setup using Cloud Build.

For file storage, we used Google Cloud Storage and wrote a custom Medusa file module to access public and private buckets. There was an issue with Medusa’s hardcoded `/uploads` folder (App Engine only allows writing to `/tmp`), so we patched it to resolve the issue.

## Notification Orchestration System – Novu & Bird

Notifications are a key part of the project. Our requirements included:

- Email notifications
- Push notifications
- In-app notifications
- Digest or delayed notifications
- Phone OTP SMS

To meet these needs, we opted for a notification orchestration system. We chose Novu due to its open-source availability and affordable pricing (compared to Knock). For push notifications, we used Expo Notifications, but we encountered issues integrating Bird (formerly MessageBird) with Novu, so we wrote a custom module to handle authentication codes while Novu handles the rest.

## Implementing Offers

Another requirement was enabling price negotiation between buyers and sellers. This involved actions like:

- Accepting an offer
- Rejecting an offer
- Making a counteroffer

We stored offer histories and used custom carts to redirect buyers once offers were approved. Notifications were crucial here since offers expire within a set timeframe.

## Messaging with Censorship

For messaging, we initially planned a simple email-like system, but the scope expanded to include:

- Messaging between buyers and sellers
- Instant message delivery
- Read receipts
- Message notifications
- Hashing of sensitive information like phone numbers, email addresses, and credit cards
- Censorship of banned words

## Error Reporting with Sentry

We used the Medusa Sentry plugin for error reporting. More details can be found in the [documentation](https://medusajs.com/).

## Shipping Provider for Printing Labels: Shippo

The requirements for shipping included:

- Basic support for fulfillment by sellers
- Automatic generation of shipping labels
- Support for multiple providers
- Customizable shipping prices via the super admin

Although Medusa has a plugin for Webshipper, we opted for Shippo due to its free entry and great documentation. Since there wasn’t a good Shippo plugin for Medusa, we wrote a custom integration, allowing us to customize workflows like setting fixed shipping prices.

## Product Approval

During the early stages, we needed a product approval flow where the marketplace could review products before they went live. Fortunately, Medusa offers these statuses out-of-the-box, so we just needed to implement filtering with Algolia and build a custom UI for super admins to review products.

## NB

This is just a surface level look at how the system was created and the reasoning behind certain decisions without going into detail of how to implement certain features, if you require more information regarding building out these systems or questions feel free to reach out via email.

small_medusa content.png

thumbnail_medusa content.png

xsmall_medusa content.png

medusa content.png

Creating a global production ready marketplace with Medusa JS

At Sparc agency, we were faced with the challenge of how do we implement oauth2 flow for one of our software, there were a few options to choose from:

- Make use of authentication and authorization as a service like Oauth0
- Write our oauth2 server with available packages like Passport for Node JS or Passport for PHP
- Use an open-source Identity solution as Ory provides.

If only the time of development were to be considered then Oauth0 would be the best choice out of the three where using packages like Passport will come last, but that was not the only concern here, so we ended up going for the last options using Ory Hydra for oauth2 and Kratos as an identity provider as our business need required a bit more flexibility also it was important that we manage our user's list in our database. 

The next challenge was then to find implement it with our stack:

- Express JS backend.
- Cockroach DB database.
- Nuxt JS login and consent screen.

Since there were not many examples on the internet yet with this kind of setup, we went back to the drawing board and came up with this:
![IMG_4936.jpg](https://slaknoah.nyc3.digitaloceanspaces.com/IMG_4936_56708761c6.jpg)
I will assume you are familiar with some oauth2 terms, and how it works, but just to refresh memory some of the terms used in this article are:

- Client: The client is the app that is attempting to act on the user’s behalf or access the user’s resources. Before the client can access the user’s account, it needs to obtain permission. The client will obtain permission by either directing the user to the authorization server or by asserting permission directly with the authorization server without interaction by the user.
- Authorization server: The authorization server is what the user interacts with when an application is requesting access to their account. This is the server that displays the OAuth prompt, and where the user approves or denies the access request. The authorization server is also responsible for granting access tokens after the user authorizes the application.
- User: The resource owner is the person who is giving access to some portion of their account. Any system that wants to act on behalf of the user must first get permission from them.
- Authorization Code: An authorization code is returned to the client after the authorization step, and then the client exchanges it for an access token.
- Access Token: An access token is the string used when making authenticated requests to the API. The string itself has no meaning to the application using it but represents that the user has authorized a third-party application to access their account.
- API: The spec refers to what you typically think of as the main API as the “resource server.” The resource server is the server that contains the user’s information that is being accessed by the third-party application.

Now, let's go through the flow:

The client initiates an oauth2 flow to the /oauth2/auth endpoint of the authorization server - Ory hydra, here the oauth2 CSRF cookie is saved which would be used later by hydra, the hydra redirects to our express backend service with the hydra_challenge, the backend service serves as an interface between Kratos our identity provider and hydra. Here I will split the flow into 3 assumptions:

- **The user is not logged in to our service and didn't ask hydra to remember their oauth2 consent**: [A]The backend service makes gets the login request info from hydra using the hydra_challenge, since the user isn't logged in and haven't previously asked hydra to remember their consent decision, we redirect the user to the login screen, this is where Kratos comes in, but before redirecting to Kratos browser login flow:
    - We generate and save an hydra_login_state session
    - Using Kratos ability to receive a return_to URL: which is the URL to return to after successful login, we set the return_to URL to our backend service with the hydra_login_state and challenge as a query parameter
    - Then we redirect the user to the Kratos /self-service/login/browser URL with the return_to param set to the URL generated in the previous step
    
    [B]Once the Kratos server receives this request it creates a login flow and saves the return_to URL to the database, Kratos then redirects to the preconfigured login screen which can we a SPA or SSR app, in our case we are using a Nuxt app which renders the login screen based on the flow-id (You can read more on how Kratos login screen is rendered here https://www.ory.sh/kratos/docs/self-service/flows/user-login/ ), here is important to submit the form instead of making an AJAX request though it might be possible to use AJAX it doesn't make sense here since there will be a redirect anyway. Once the user is logged in, Kratos returns to the return_url previously set also with a **set-cookie** header including the ory_kratos_session cookie - this tells the backend service who is logged in.
    
    [C]Now the backend server does another check, where it needs to check if a user is logged in(and they are because there is a Kratos session cookie available) also it looks for the hydra_login_state in the query and checks it against the one in the cookie that was saved on the first hit before redirecting to Kratos, if they are same and the user is logged in, we ask hydra to acceptLoginRequest with the using the challenge, and the user from Kratos session.
    
    [D]Next action would be to redirect to the hydra URL returned by the request, here hydra ensures the cookie set at the beginning of the flow is still set it then returns to the preconfigured consent URL which is the backend service again in our case, here we get the consent information with the consent_challenge provided, if the user hasn't previously asked hydra to remember their consent decision, we redirect to our Nuxt app consent screen with the challenge, client, user, CSRF token(generated using csurf package) and requested scope.
    
    [E]The Nuxt app renders the consent screen, the user authorizes the client, where we can use AJAX request, to our backend service with the granted scope challenge and CSRF token, we ask hydra to acceptConsentRequest, hydra returns the client's callback URL set by the client, with the authorization code, we then redirect back to the client app where the client can then exchange the authorization code for the access token and refresh token (if applicable) and get user info or perform other operations on our API using the access token.
    
- **The user is logged in to our service but hasn't previously asked hydra to remember their oauth2 consent: [A]**In this case the flow starts as before but on fetching login request with the challenge provided, hydra detects the user is logged in as was set previously and returns the consent URL as the redirect URL, if not ( i.e they are logged in to our service but haven't consented to the client oauth2 request) the backend service performs the three steps in [A] of the previous point to generate the Kratos URL, and once Kratos receives this request it detects a valid session cookie that the user is logged in, so it doesn't bother rendering login state, it just redirects to return URL, which is the backend service URL that includes the hydra_login_state and hydra_challenge as before.
    
    [B]From here the flow continues from [C] - accept login request, then [D] - initiate consent to [E] - accept consent request, of the previous point.
    
- **The user is logged in and had previously asked hydra to remember their oauth2 consent:**
    
    With this, the user wouldn't experience any page redirect on clicking login with our service. The flow is the same as the [A] of the previous point since the user is logged in, but is different when it comes to the consent step.
    
    Once the backend service is hit with the request it performs [C] from the first point which is to accept login request but when its time to perform [D] hydra detects the user had previously checked the remember consent option, so we can skip consent, we accept consent request and redirect back to the client app with the authorization code where the client can then exchange the authorization code for the access token and refresh token (if applicable) and then go-ahead to get user info or perform other operations on our API using the access token.
    
    And that pretty much wraps it up.

large_fingerprint-authentication-2021-08-26-19-52-29-utc.jpg

medium_fingerprint-authentication-2021-08-26-19-52-29-utc.jpg

small_fingerprint-authentication-2021-08-26-19-52-29-utc.jpg

thumbnail_fingerprint-authentication-2021-08-26-19-52-29-utc.jpg

xsmall_fingerprint-authentication-2021-08-26-19-52-29-utc.jpg

fingerprint-authentication-2021-08-26-19-52-29-utc.jpg

Setting up production-ready ORY Hydra and Kratos oauth2 flow using Nuxt js for rendering login and consent screen

Let's start with a little background. Just like you probably do, I love to develop things for the web with up to 5 years of experience as a backend developer using  PHP, Laravel, Elastic search, Docker(even though it's not a backend technology), and 3 years as a frontend dev, working mainly with Vue as my framework of choice (for the sake of the length of this article I won't get into why I chose Vue over other nice frameworks out there).

I recently took on a new job where part of my responsibilities includes the development of a web app and collaborating with the API development, and since the API is Node JS I have no other option than to finally give in to learning how it works.

So as anybody will do, I went ahead to Udemy and obtained a Node Js course, about 20% into the course I noticed something, apart from the fact that they were basically teaching how the web works, I actually understand everything being taught, then it finally came to me what I have been seeing all over the internet about Node Js, its simply put javascript running server-side, since I know javascript, it all makes sense now.

After that discovery, I stopped the course and decided to learn by trial *(btw I wrote an article about how to leave the tutorial hell here*) instead, by building a simple API as that is what I will be doing most, from there, actually writing API went pretty smooth with help of documentation and articles on the internet. Then I went ahead trying two frameworks (Express and Fastify) for the same project.

Here are some of the things I noticed in the course of development especially in comparison with where I am coming from. 

- On the language level, Node is quite cleaner than PHP thanks to javascript, though that might be from my preference for javascript over all other programming languages.
- When it comes to speed, Node JS is faster
- PHP in my opinion is more powerful so if your project needs a lot of processing in the backend you might want to opt for that
- On the framework level, with the most popular frameworks available in both languages, Laravel, and Express, Laravel unlike Express is opinionated meaning it encourages you to follow a particular structure while express gives you the freedom to structure your application the way you want to. With that comes some disadvantages too though because if you are not careful things can get really messy so I checked some examples online of how people structure their apps. This repository was especially helpful: https://github.com/santiq/bulletproof-nodejs/

With that I was up and running with node, I know there is still a lot to learn but I believe the basics have been covered.

This is simply evidence or an answer to the popular question "Which language should I learn first?", It doesn't really matter what you begin with as long as its not some really old one, just get started with one you find convenient to learn basic programming principles from which you can then port/move to other languages as you wish.

large_african-american-coder-employee-programming-busine-2021-12-09-02-41-56-utc.JPG

medium_african-american-coder-employee-programming-busine-2021-12-09-02-41-56-utc.JPG

small_african-american-coder-employee-programming-busine-2021-12-09-02-41-56-utc.JPG

thumbnail_african-american-coder-employee-programming-busine-2021-12-09-02-41-56-utc.JPG

xsmall_african-american-coder-employee-programming-busine-2021-12-09-02-41-56-utc.JPG

african-american-coder-employee-programming-busine-2021-12-09-02-41-56-utc.JPG

My experience moving from PHP to Node JS in one week

My checkbox towards becoming a better developer/programmer

- Don't get too comfortable: Whether it’s with a language, framework, or any other technology, try not to get too comfortable by either finding more information to get better using the tech or considering something new.
- Learn the basics: Once you understand the basics i.e how your language work under the hood, using any framework or library based on it becomes easier, for example learning the basics of Javascript will help you with using its frameworks like Vue, React, Angular e.t.c. and understanding core HTTP request-response cycle helps a long way using any backend technology.
- Take a break but with caution: We are not clocks, learn to take a break every now and then to regenerate and avoid stress, would advise though that it's not too long, so you don't have to relearn a lot when you return.
- Share your knowledge: As much as possible whenever you have the opportunity to, share what you know it helps a long way to have a better understanding.

large_tim-gouw-1K9T5YiZ2WU-unsplash.jpg

medium_tim-gouw-1K9T5YiZ2WU-unsplash.jpg

small_tim-gouw-1K9T5YiZ2WU-unsplash.jpg

thumbnail_tim-gouw-1K9T5YiZ2WU-unsplash.jpg

xlarge_tim-gouw-1K9T5YiZ2WU-unsplash.jpg

xsmall_tim-gouw-1K9T5YiZ2WU-unsplash.jpg

tim-gouw-1K9T5YiZ2WU-unsplash.jpg

How to get better as a developer/programmer

fd60238a-f68c-4f48-b3a7-32a8eb5a20cf_ezgif.com-gif-maker+(1).avif

Ultimate Arcade (Cyberworms)

Developed the 2D frontend of the play to earn project using Javascript and WebGL

small_nexaloom-ezgif.com-optimize.gif

thumbnail_nexaloom-ezgif.com-optimize.gif

xsmall_nexaloom-ezgif.com-optimize.gif

nexaloom-ezgif.com-optimize.gif

Nexaloom

Nexaloom is an Appointment Scheduling and Business Software side project of mine

small_ScreenRecording2024-08-08at20.36.54-ezgif.com-video-to-gif-converter.gif

thumbnail_ScreenRecording2024-08-08at20.36.54-ezgif.com-video-to-gif-converter.gif

xsmall_ScreenRecording2024-08-08at20.36.54-ezgif.com-video-to-gif-converter.gif

ScreenRecording2024-08-08at20.36.54-ezgif.com-video-to-gif-converter.gif

*angia

*angia is a viewer interaction platform designed to allow creators to engage with their audience
on different streaming platforms like Twitch, features interactions like TTS, Al voice, Soundbits,
Al Chat etc. Developed advanced Ul for both Creators

small_ScreenRecording2024-08-08at20.49.54-ezgif.com-video-to-gif-converter.gif

thumbnail_ScreenRecording2024-08-08at20.49.54-ezgif.com-video-to-gif-converter.gif

xsmall_ScreenRecording2024-08-08at20.49.54-ezgif.com-video-to-gif-converter.gif

ScreenRecording2024-08-08at20.49.54-ezgif.com-video-to-gif-converter.gif

TOST

Developed a transit management software (full stack) for creating and managing GTFS industry
standard file for bus routes used by major transit companies in the United States.

Full-Stack web and app developer,
based in Birmingham, UK

Latest works

Ultimate Arcade (Cyberworms)

Nexaloom

*angia

TOST

Web and mobile stacks

Latest articles

Creating a global production ready marketplace with Medusa JS

Setting up production-ready ORY Hydra and Kratos oauth2 flow using Nuxt js for rendering login and consent screen

My experience moving from PHP to Node JS in one week

How to get better as a developer/programmer

Full-Stack web and app developer, based in Birmingham, UK

Latest works

Ultimate Arcade (Cyberworms)

Nexaloom

*angia

TOST

Web and mobile stacks

Latest articles

Creating a global production ready marketplace with Medusa JS

Setting up production-ready ORY Hydra and Kratos oauth2 flow using Nuxt js for rendering login and consent screen

My experience moving from PHP to Node JS in one week

How to get better as a developer/programmer

Full-Stack web and app developer,
based in Birmingham, UK