FishtankVR for The Witcher 3

[AwesomeO]

Hey,
I don't know if this is the right place, but since I've been lurking around here for quite some time and 3DVision mostly is responsible for me working on this mod ata ll, I wanted to drop the link here.
So I've been working on a FishtankVR implementation for The Witcher 3 as a proof of concept for combining head-tracking with perspective correction and stereoscopic 3D, and it turns out that it works really great.
Let me know what you think!

https://www.youtube.com/watch?v=dc2QfUrX9hs

[costiq]

AwesomeO, this is A.W.E.S.O.M.E.!!!
Amazing work.

Please tell us more. I want this!

[ToThePoint]

Yep, I'd say you've most definitely come to the right place. Lol! Very nice indeed. As someone that's used TrackIR for a very long time now in order to further enhance my monitor based stereoscopic gaming experience, this is very interesting. You may perhaps be aware that a company called Dimenco have produced a monitor that combines what you've presented in your video, in combination with stereoscopic 3D and as well as haptics.

They've brought a developer's version to market, which being incredibly expensive, is simply out of reach for the average 3D gamer unless they happen to command a six figure salary or until Dimenco possibly bring out a retail version in the next year or three. But who knows.

Nevertheless this is a very intriguing development, and if you're ever in a position to make this available either for free or indeed for purchase, then I'm sure there would be a lot of interest here.

At least theoretically, the head tracking device and software that you're currently using and developing, along with a TrackIR head tracker and a 3D Vision emitter, all of a sudden means that the available real estate on top of one's monitor is suddenly at a real premium. At least for me anyway. It would be great if you could keep us posted.

[russellk]

That looks great!

Is that a Ps3 eye camera on top of the monitor?
I bought a couple to use for facetrack/NoIR, I can't remember the name now but I never got round to using them. Not even sure what models I have or if I need to remove the IR filter, as I bought a trackIR system in the end but never used that either and keep meaning to put it on ebay.
I think another complication with the PS cameras was that the drivers were free when I had Windows 7 but then got put behind a paywall?

Anyway, well done!

[AwesomeO]

Yep, it's a PS3 eye cam up there with IR filters removed. It's pretty neat as it gives you 120fps in 320x240 (which is totally sufficient for tracking some IR dots). I had no issues getting it to run on Win10 nor did I have to pay for any drivers.
I've also started this with FaceTrackNoIR but the bad depth tracking + wearing big 3DVision glasses really messed up the face recognition...

So currently I'm using Opentrack with a DelanClipFusion (wireless IR tracking device) mounted to my 3DVision glasses which works very good but could definitely be improved. I'm thinking about mounting just 2 IR Diodes onto the 3DVision glasses somehow.

The future of this project heavily depends on how much time i can invest into this (and that's not much with a job and 2 kids ).
There are still some dependencies that I'd like to get rid off, so that the whole thing is just one package.
Also proper and easy calibration is a tough problem, but there's already something there so anyone who's willing to invest some sweat and tears could live with that.
It would be a perfect addition to HelixVision though, as headtracking and screen dimensions are already available there.

And as mentioned in the video this is currently more a proof of concept, and I'll yet need to find out how to apply this to a wider range of games.
This will probably take me on another interesting journey

... and the cancelled official support for 3DVision was a real letdown... Not really a motivator, so I really hope the 3DVision community finds a reliable way to keep things going (kudos to everyone involved with that!)

@ToThePoint: Dimenco looks really interesting. I wonder how they provide the two images to the eyes... Looks like it only works within a tiny specific range like with Nintendo3DS/3DTVs

[ToThePoint]

@AwesomeO

Sorry for the late reply. Yeah, I can imagine this concept being even more of a natural enhancement of VR-based HelixVision. It might be worth bo3b taking an interest in this project of yours at some point. Haven't been console gaming for years, so was unaware that Playstation even had eye tracking peripherals available that far back. I see that web-based outlets have quickly snapped up used PS3 Cameras, to then sell them on with the IR filter removed, as linked below. Mounting clips available too. All very convenient.

https://delanengineering.com/shop/PS3-C ... -p50254011

I guess since I've been content enough with TrackIR, yet again I was unaware of open source software like Opentrack. Checked out EDTracker's website and videos as well.

Assuming that you've not already researched yet, Dimenco's glasses-free 8K monitor/devkit has a lenticular array incorporated within it, which has a 60 degree viewing angle, and until I saw your own demonstration, that was the first monitor that I'd seen where you got the impression that you were actually looking through a window, rather than just at a 2D surface. 3D Vision already does that to a great extent of course, but this feature just adds even more to the realism.

You may not have the haptics sorted out...just yet , but leaving that firmly aside, your approach is approximately 430 times cheaper in Euros, to get the same visual effect that Dimenco's devkit currently does, and I reckon that that's pretty good value!

Sounds like you've got your hands full with plenty of other commitments as well, so I really don't think there's any rush here! Thanks though for bringing this to our attention in the first place. It's definitely an eye opener.

[russellk]

@ToThePoint

A few yeas ago (I can't even remember how long now!) a couple of 'homebrew' alternatives to TrackIR appeared. The PS Eye camera was favoured as it's capable of 120hz input and is cheap and readily available. The Playstation itself never had any eye tracking that I'm aware of, the camera was used to track the light orbs on the PS Move controllers.

I believe there were at least 2 different models of camera, most required the removal of the IR filter but there might have been one where you didn't have to? It looked a bit fiddly and I never got round to doing anything with mine anyway, as I purchased a TrackIR in the end.

You're probably aware that TrackIR is/was quite expensive and I seem to remember there being a bit of a fuss about the headtracking protocol itself, hence the alternatives appearing.

[ToThePoint]

@ToThePoint

A few yeas ago (I can't even remember how long now!) a couple of 'homebrew' alternatives to TrackIR appeared. The PS Eye camera was favoured as it's capable of 120hz input and is cheap and readily available. The Playstation itself never had any eye tracking that I'm aware of, the camera was used to track the light orbs on the PS Move controllers.

I believe there were at least 2 different models of camera, most required the removal of the IR filter but there might have been one where you didn't have to? It looked a bit fiddly and I never got round to doing anything with mine anyway, as I purchased a TrackIR in the end.

You're probably aware that TrackIR is/was quite expensive and I seem to remember there being a bit of a fuss about the headtracking protocol itself, hence the alternatives appearing.

You're quite right of course. The PS 'Eye' Camera is just a head tracking device, but Dimenco's devkit actually does do eye tracking, and yet AwesomeO is getting the exact same result. That's really interesting.

Regarding the price, yeah, I bought another one just recently, only they're a little more difficult to source nowadays. To my surprise, Naturalpoint are still producing new game profiles and further updates. Almost exclusively in my experience, you don't have to set up TrackIR to work for a specific driving game for instance. It just does it automatically. Just jump in the car and off you go. I love the fact that you can actually lean in either direction when turning into long fast curves, in games like Project Cars, Assetto Corsa and DiRT 3. So I still feel like I'm getting my money's worth.

Anyhow, thanks for that correction.