Stan Larroque's Blog

Category: Lynx

  • Prototyping any stereo camera with StereoCam

    This post is for any person in AR/VR/drones and beyond who want to quickly prototype with stereo cameras.

    Our StereoCam prototyping platform

    At Lynx, we are big on passthrough for Mixed Reality and thus we always had a need to test various setup of stereo cameras. We even built a passthrough MR headset with 140dB+ HDR cameras (probably not the HDR you think about).

    In order to test various lenses and sensors when tinkering future HMD configurations, I built as a weekend project a prototyping platform for the team where the goal was to be as simple and effective as possible to have quick results.

    The result is a platform built on top of the latest Raspberry Pi 5. The Raspberry Pi 5 has 2 dedicated MIPI ports, and all the bells and whistles you can expect from a small yet powerful ARM computer. There is even a 16Go RAM version now, pretty cool.

    What you’ll need to assemble this:

    The casing is separated in 3 volumes (CamHolder, Case, Lid). You can 3D print these with standard PLA very easily.

    CamHolderDownload
    CaseDownload
    LidDownload

    The CamHolder was designed to hold the High Quality Camera from Raspberry Pi. It is a camera module based on a high-res Sony IMX477 sensor. So you can add two of these to the BOM.

    The case is designed to hold the Raspberry Pi 5 and its active cooling add-on. If you’re just streaming the cameras over the network to do processing somewhere else, then the whole system should only consume 5-6W, which could be fine without a cooler. Otherwise the active cooler is greatly encouraged.

    The case supports all RPi interfaces and its active cooler.

    To connect the MIPI cameras to the Raspberry Pi, you’ll need this kind of FPC: https://www.digikey.fr/fr/products/detail/raspberry-pi/SC1128/21658272. It also exists in 300mm and 500mm length.

    You also need to pick the right lens for your camera. Here is a vendor with a conveniant tool to pick the right FOV based on your resolution requirements.

    In Lynx use-case, we dump the RAW frames over the network so we can process them later and on a more powerful computer. We use the Raspberry Pi strictly as a “networked capture card”. The board is connected over Ethernet and we can start streaming the cameras with simple commands thanks to the Raspberry Pi incredible software support for cameras. You should find there your preferred ffmpeg/streaming command lines and if you don’t, you can always dig deeper with libcamera.

    Now you have in your hands a very capable stereo camera setup that can do a little bit off onboard processing (not any heavy AI stuff, but OpenCV distortion correction etc is largely feasible). You could even mount it as a payload on a drone and do some cool stereo FPV!

  • The Next Step (2/2)

    Android XR banner featured on Google announcement’s post

    Finally, Google went public with Android XR, after years of working on this new paradigm for the Android ecosystem.

    All standalone XR headsets on the market are running currently with their own customized version of AOSP, the open-source part of what people know as “Android”. On top of that, Google just released developer tools, optimizations, perception algorithms and various other software solutions all dedicated to XR. Which I believe is great news for the overall XR market.

    Below is a Road To VR article showcasing some projects who were onboarded with Android XR. Lynx is very much part of this group.

    Lynx Confirms Android XR For Next Headset, Sony & XREAL Also On-board for Google’s OS
    Ziad Asghar, Qualcomm SVP of XR during the Google announcement event in NYC.

    The announcement (video here) was centered mostly around the high-end MR Samsung headset. I have to say I was quite amazed by the demo and the device itself. Think of it as the Vision Pro but without the front display and with a more rigid strap. The passthrough was pleasant, mostly because of the low-latency of the micro-oleds panels (compared to a LCD display, they don’t need to wait for the full frame to be rendered to display an image). The MTF of the lenses was quite low in periphery for a pancake lens, but the units they were showing to the press are “early units” and some changes are to be expected.

    Now for Lynx, it’s great to be able to talk a bit more about this collaboration with Google and Qualcomm. We are adopting Android XR as an OS compatible with our future headsets. The Lynx-R1 will not be compatible with AXR, but I expect all our future products to be AXR-compatible.

    The hardware team is working hard on a new product, and you will hear more later in 2025. The software team is still doing R&D and OS work on the side of our AXR integration, for specific customers like in Defense or the medical field, and for some users who will still want something different.

    Lynx is basically doing in XR the opposite of Blackberry in the mobile market. Blackberry was killed by the iPhone, but it was also mostly killed by refusing to adopt Android and thus lacked an ecosystem on the long-run. On our end, we are aligning Lynx with a great partner and future ecosystem like Android XR, while also having our own solution for some specific needs.

    Edit (21/12/24): for those of you who understand French, here is an interview I gave on French television from New York reacting to the Android XR event.

  • The Next Step (1/2)

    Tomorrow, exciting news will occur for everyone in the AR/VR (still small) world. Lynx is involved and some pieces of something we’ve been working on for a few months will be revealed.

    I can’t talk about anything that will occur tomorrow or the partners involved, but I expect a round of tech news to cover it extensively.

    No matter what will happen and what you can learn/speculate from it, these are my words:

    Lynx is still committed to the same mission: unlock the best Mixed Reality experiences with amazing devices and OS-level integration for spatial computing.

    We are not acquired, nor did a new fundraising. The team is still in place, and still working on exciting new projects and supporting the existing Lynx-R1.

    With the latest hardware, optics, and perception sensors and algorithms available now, Mixed Reality is at a level I could only dream about when I started Lynx.

    There are more software engineers working at Lynx than hardware engineers (approx. 2/3 SW, 1/3 HW), and they are still working on our OS based on AOSP, from drivers and kernel patches all the way up to the application layer and OpenXR SDK.

    For the R1, we are shipping now the last hundreds of devices going to our community (you should see the shipping emails before the end of this week), and will refund the few who asked, as previously declared. We needed a bit more time than anticipated after this summer as the company faced some challenges in 2023 and 2024.

    Now for what is coming ahead, the team and I can only contain our excitement for now, but we believe Lynx has now passed a crucial inflection point, both in resources/engineering available and means to really build in quality and quantity at the level of our initial ambition.
    Please remember that our total budget for the last 4 years is approximately 4 hours of budget at Meta (1B$/month, divided by 720 hours was our yearly burn rate lol).

    I can’t wait to share more early next week with you all. Expect a longer blog post.

    Stan

    _________

    Unrelated note: I’m using this post to also signal that I probably won’t be posting on Twitter/X anymore, I generally don’t use social media and grew tired of it all, and I’ll just use this website more if I need to share things.

    Unrelated note for the French here (no subtitles available for now): I talked about AR/VR in this video last month, mainly reacting to Meta Orion.

    I’ll be speaking at SPIE AR/VR/MR conference at the end of January.

  • Lynx Original Hardware Team

    From left to right, Hugo Artarit, Laurent Chabin, Stan Larroque, Didier Lasne, Corentin Lefebvre, Etienne Denoly

    As I’m preparing a Lynx press release for Monday, we took some pictures at the office.

    This is the original hardware team at Lynx that brought the Lynx R-1 to life.

    From left to right:

    • Hugo Artarit, industrial designer
    • Laurent Chabin, electrical engineer
    • Stan Larroque, group leader
    • Didier Lasne, mechanical engineer
    • Corentin Lefebvre, optical engineer
    • Etienne Denoly, industrial designer

    Of course, not in the picture are people working at Compal, Qualcomm, Limbak and Ultraleap that helped on the work related to the hardware at different points in the development.

    I’m saving this photo here because this is probably the last time we’ll work together in this setup because Lynx is growing fast and we’re hiring a lot, and I’m very proud that we achieved with such low resources.

    I’ll do another photo with the software team of course!

    Here is the photo of the full Lynx team that created Lynx R-1:

    Lynx Team November 2021 in the Paris Office.
    Left to right: Jean-Marie Vaidie, Didier Lasne, Cedric Morel Francoz, Marc Piuzzi, Etienne Denoly, Hugo Artarit, Laurent Chabin, Stephane Amoa, Stan Larroque, Guillaume Le Fer, Chouki Hadri, Geoffrey Marhuenda. Missing in the picture: Corentin Lefebvre, Guillaume Chevalier.

    If you want to join this team, contact me.

  • Throwback: the first visual demo of Lynx

    This video clip is the first visual demo of my Mixed Reality system, it was not named Lynx, and it wasn’t even named at all! It was featuring a simple helicopter model that land on a surface in front of you.

    The viewer was an Oculus DK2 and the program was running in Javascript thanks to WebVR on Firefox. There already was two cameras for correct stereoscopic view, fetched with the getUserMedia API.

    It’s funny because at this time, it feels like I opted for the worst stack for the proof-of-concept: it was running on a Mac and in a browser, the whole thing written in javascript. The definition of the cameras was awful, no distortion was corrected, it was all fixed with duct tape and cardboard.

    But it worked! It was in September 2016, I still have the source code. And I’m really proud of the version we have today.