Apple Rejects iOS App For Using MoltenVK Vulkan, Alleged Non-Public API

[onicsis]

I don't own an iPhone, never programmed an iOS device. But toward iOS, Apples seems to be way to restrictive meaning iOS devs can't compete in the real sens with Android and their future hanging at the mercy of an illuminated and all-knowing dictatorship. At least Android AOSP is opensource, Google has monopoly through legal agreements with manufacturers to guarantee their access to Google Play and other services. Google has no real interests to stop Amazon to develop their one tablet, but most compliant with Google standards and requirements. In contrast, Amazon can offer its own services on their own Android devices.

No one has the legal right to sell iOS phone or tablets other than Apple.

Usually an Android has the builtin option to select another application store, different from Google Play even to install what applications he/she like or need it.

Also Microsoft has relied on others for producing Windows Mobile devices and has open specifications for design, engineering and manufacturing.


A few years ago a french company sued Apple to open up iOS for other browser engines.

Nexedi is suing Apple under French law in hopes it can force Apple to allow rival browsing engines onto the iPhone.

Although Apple allows rival browser apps, such as Google’s Chrome, on to the iPhone, they all have to use Apple’s Web rendering engine. That means the ability to draw on the latest Web standards is is limited to whatever Apple decides to include.

That means some newer technologies, like the WebM video standard and the WebRTC protocol for real-time communications, can’t be made to work in an iOS browser even though they work in nearly all other browsers.

[Weasel]

It's most definitely NOT an Apple thing. The Linux kernel has a private API, Qt has a private API etc, Microsoft has private APIs...

You're missing the point, it's not what YOU call them, it's what they call them.

AFAIK, Linux calls it "in-kernel" interfaces or internal. I've also never seen the words "private API" used in MSDN (Windows).

For example: https://en.wikipedia.org/wiki/Linux_kernel_interfaces

The Linux kernel provides several interfaces to user-space applications that are used for different purposes and that have different properties by design. There are two types of application programming interface (API) in the Linux kernel that are not to be confused: the "kernel–user space" API and the "kernel internal" API.

I can't find even the word "private" (except for something unrelated) on that page. So to me it's an Apple (or mobile crap) term.

[billhollings]

We are in contact with the app developer about this.

MoltenVK does NOT use any private or undocumented API’s to work it’s magic. Everything comes from Apple’s public frameworks.

IOSurface as a framework on iOS seems to be in some strange middle state...as in recently transitioned from a private to public API…but it is now a documented, public API framework on iOS. As mentioned in an earlier post…we do use one deprecated IOSurface parameter…but even that is a publicly declared parameter in Apple’s public headers.

If MoltenVK does turn out to be offending something in using IOSurface on iOS…it is certainly unintended…and is easily remedied.

Looking beyond this administrative hiccup...MoltenVK always has...and always will...remain committed to a full implementation using ONLY Apple's approved, documented, and publicly available API's.

Bill Hollings
The MoltenVK Team

[natbro]

Why's the GLSL step necessary there, btw?

Ah, it's not always going all the way through each of those transpiles, it's just a convenient way to talk about all the ways things get to SPIR-V and eventually to MSL.

[natbro]

https://github.com/KhronosGroup/Molt...KImage.mm#L328
There is the line that caused rejection. Just read the comment about. It is deprecated (even Clang warning is disabled) and states about interprocess communication which does not exist for iOS. MoltenVK developers just need to remove that flag for iOS and that's it.

Upon closer inspection, we think we need to pull all IOSurface references from MoltenVK for iOS since beyond the deprecated cross-process flag the framework is private prior to iOS 11.0 and MoltenVK can deploy back to iOS 9.0. We've got a PR to do that (https://github.com/KhronosGroup/MoltenVK/pull/194) and working with the developer. A better solution will be having IOSurface support in iOS 11+ builds of MoltenVK. Even if they can't be used on iOS for cross-process sharing they are useful for allowing you to composite with or source from camera and video sources (AVFoundation pixel buffers) without needing to duplicate expensive texture resources.

[V1tol]

Upon closer inspection, we think we need to pull all IOSurface references from MoltenVK for iOS since beyond the deprecated cross-process flag the framework is private prior to iOS 11.0 and MoltenVK can deploy back to iOS 9.0. We've got a PR to do that (https://github.com/KhronosGroup/MoltenVK/pull/194) and working with the developer. A better solution will be having IOSurface support in iOS 11+ builds of MoltenVK. Even if they can't be used on iOS for cross-process sharing they are useful for allowing you to composite with or source from camera and video sources (AVFoundation pixel buffers) without needing to duplicate expensive texture resources.

Actually I didn't know that IOSurface was exposed in iOS 11. But as far as I remember, buffers from camera and video sources are already using global IOSurface (already saw that flag when I was writing one app and deep inspecting CVPixelBuffer objects received), because it never was a problem to draw those pixelbuffers directly without copying. The reason about this flag is to create your own surfaces or pixel buffers that can be shared among processes but it is obviously restricted by Apple on iOS devices. For example, if you needed a CVPixelBuffer that is backed by IOSurface, you could just provide an kCVPixelBufferIOSurfacePropertiesKey with empty dictionary ( https://developer.apple.com/document...?language=objc )

[the_scx]

You make no sense. With that logic, nobody would ever pursue a proprietary API, because they could just as well use open one, if all they cared about is having more apps run on it.

Exactly, just like in Windows system. No game developer cares about DirectX here, because we have OpenGL, which is a multi-platform, open source standard. Oh, wait. It does not work as you expected.

With MoltenVK, very very very few devs will develop for Metal.

Yeah...
https://twitter.com/Housedakitopal/s...19710445649920 - "MoltenVK's job is to convert Vulkan games to Metal, which isn't a process we require. We already work directly with Metal, Apple's native API." & "We'll always do what's best for each of our supported platforms, no matter what. If using different technologies will give our macOS and iOS-using followers the best possible experience, then that's what we'll do. The same goes for Linux."
https://www.phoronix.com/scan.php?pa...l-No-Linux-API - "Feral announced their macOS port would be making use of Apple's low-level Metal graphics API."
https://www.phoronix.com/scan.php?pa...Vulkan-H1-2017 - "Sadly the upcoming Deus Ex Linux port will only be using OpenGL while the macOS port is using Metal and the Windows game supports Direct3D 12."
https://www.phoronix.com/scan.php?pa...b-Raider-Linux - "the macOS version will obviously use Apple's Metal graphics API"

Moreover, you stayed in the 90s (maybe even earlier), when everyone created their own game engine. This situation no longer exists. Nowadays, developers in most cases choose a third-party game engine, based on their needs:
1. For very specific games you can use dedicated engines, for example: Twine + SugarCube (story format) for text-based novels (interactive, nonlinear stories), RenPy for Visual Novels, RPG Maker for 2D RPGs, PaC-DK (The Point & Click Development Kit) for point-and-click adventure games, etc.
2. For simple 2D games you can use a relatively simple 2D engine, for example Cocos2D or Construct 2 + Tiled (level editor).
3. For medium-range games, you can use Unity 3D + Fabric/FMOD/Wwise (sound engine) + PlayMaker (visual scripting).
4. For more demanding titles you can choose Unreal Engine 4 or maybe even CryEngine (less popular, with worse documentation, generally harder to use).
Most mobile game developers usually go by route 2 or 3. Of course, developing your own engine for really demanding titles still makes sense (e.g. Rockstar Advanced Game Engine, Frostbite or AnvilNext), but it's a niche, especially on mobile market.

As for Unity, the choice of graphic API comes down to selecting one option from the menu.

https://docs.unity3d.com/Manual/UsingDX11GL3Features.html

https://docs.unity3d.com/Manual/class-PlayerSettingsAndroid.html

https://docs.unity3d.com/Manual/class-PlayerSettingsiOS.html

Of course, in practice it is more complex and not always so simple. However, in general, developers are not attached to specific API. For example, Dreamfall Chapters began as the DX9-based title, but currently supports also DX11 for Windows and OpenGL for Linux.

They don't want that, they are jealous on Microsoft's DirectX and want to pull something similar to "lock in" people to their crap ecosystem. Look at "DirectX-exclusive" apps for what I'm talking about. That's what they want to do, but it's never gonna happen, because their ecosystem is pure garbage.

DirectX was successful on the Windows platform not because MS has forbidden the use of other APIs (what never happened), but because it better met the needs of game developers.

It should be emphasized that DirectX is a much more comprehensive solution compared to OpenGL. It offers not only the 3D rendering API (Direct3D), but also:
- DirectDraw/Direct2D for 2D graphics
- DirectWrite for fonts
- DirectCompute for GPU Computing
- DirectSound/DirectSound3D/DirectMusic/XAudio2/XACT3 for audio
- DirectShow/DirectX Media Objects for streaming objects such as encoders, decoders, and effects
- DirectPlay/Games for Windows Live/Xbox Live (well, not really part of the DX, but also included in the MS ecosystem) for communication over a LAN network or Internet
- DirectInput/XInput for handling input devices, especially game controllers
- DXGI for enumerating graphics adapters/display modes, selecting buffer formats, etc.
Of course, not all of these APIs are really interesting for game developers (or rather the creators of game engines). Although APIs like XInput are still important, comprehensiveness of DX was relevant mainly in the 90s and the beginning of the 21st century (in the past, games actually used DirectShow for video playback and DirectPlay for network communication, especially via LAN).
Therefore, let's focus on the graphics API. Even here, the advantage of DirectX over pure OpenGL is huge. Just one word: D3DX (Direct3D Extension) and its counterparts for DX11/DX12. DirectX has undergone a long evolution and it's hard to describe it as a single API even if we narrow the field to DX9-DX12, so let's compare what we have in D3DX (from DX9-DX11, but mainly DX9-DX10) with its counterparts for DX12 (this also applies to DX11):
- general helpers (2D sprites, bitmap based text rendering, primitives, geometric primitives/common shapes like cubes or spheres, basic support for models/meshes, and even basic effects/common shaders) from D3DX → DirectXTK (DirectX Tool Kit for DirectX 11)/DirectXTK12 (DirectX Tool Kit for DirectX 12)
- mathematical functions (linear algebra operations on vectors and matrices) from D3DXMath (D3DX*Vec*, D3DX*Matrix*) → DirectXMath + SimpleMath (as wrapper)
- spherical harmonics math from D3DXMath (XMSH* functions) → SHmath (as an add-on for DirectXMath)
- geometry functions (Mesh) from D3DX (D3DX*Compute*, D3DX*Mesh*) → DirectXMesh
- texture functions from D3DX (D3DX*Texture*, D3DX*GetImage*) → DirectXTex
- texture atlas from D3DX (D3DX*UVAtlas*, D3DX*ComputeIMTF*) → UVAtlas
- effects (FX) from D3DX (D3DX*Effect*) → Effects for Direct3D 11 (FX11)
- HLSL functions from D3DX (D3DX*Compile*, D3DX*Shader*) → D3DCompile
As you can see, D3DX has undergone serious modularization. It should be mentioned, that DirectXTK is de facto a collection of helper classes. In addition to the aforementioned things, it also provides API for audio (built on top of XAudio2), handling input devices, including gamepads (based on XInput), etc.

In pure OpenGL, you have literally nothing like that. You have to build everything yourself or at least assemble your own kit from 3rd-party components. More specifically, you must take care of the following things:
- math
- meshes
- textures loading
- texture atlas
- fonts
- shaders loading
- window handling and integration with the system UI
- debugging capabilities
Moreover, you must also deal with support for input devices (keyboard, mouse, gamepad), video playback, audio, network, etc. And of course, there are quite good components. For example, you can use GLM (OpenGL Mathematics) for math, GLI (OpenGL Image) for textures, Assimp (Open Asset Import Library) for assets loading, and GLFW for creating windows, contexts and surfaces as well as receiving input and events. The same applies to the things not related to graphics. For example, you can use OpenAL for audio.
The problem is that it is not as unified as DX and the future of some components is not entirely certain. Also, not everything is super mature. For example, just a few years ago, DevIL (IL+ILU+ILUT) was quite a popular library for image loading, and then suddenly its development stood still around 2009. This project tried to be reactivated as ResIL, but not much of it came out. A similar thing happened GLUT (OpenGL Utility Toolkit) - a utility library for managing windows containing OpenGL contexts. This toolkit had many limitations and its development stopped around 2000. It was essentially replaced by the FreeGLUT. There was also attempt to create OpenGLUT. Now we have GLFW (Graphics Library Framework), which presents a different approach... Eh, you know what I mean?

We also have things like SFML (Simple and Fast Multimedia Library) and SDL (Simple DirectMedia Layer). Well, there is a reason why they have "Simple" in their names. Let's focus on the second one. This library was created by Sam Lantinga while working for Loki Software. It was intended for porting games from Windows, not as a full-fledged framework that can be compared to DirectX. Moreover, its development has practically stopped after the collapse of the company. The development started again after hiring Sam in Valve (AKA Loki 2.0 :P). Unfortunately, it was too late. Do not get me wrong. SDL is a pretty good piece of code, but the gaming market has changed significantly over that time. Those who would benefit from this have already chosen 3rd-party engines or actually they are developing their own from scratch. In the latter case, SDL can be useful, but rather for basic stuff, like windows and input handling.

That's what I said here about OpenGL also applies to Vulkan. Actually, there are no components dedicated only to this API. So don't be surprised if you find out that some Vulkan-based game uses GLM (OpenGL Mathematics) for math, GLI (OpenGL Image) for texture loading and GLFW (Graphics Library Framework) or SDL (Simple DirectMedia Layer) for basic stuff like windows handling.

Coming back to the topic, Apple created the Metal API not because they hate open source standards, but just because they had no other choice. GLES (OpenGL ES) was not enough for them, and Vulkan was not even planned at that time. They certainly put a lot of effort into it, providing a comprehensive solution, starting from the low-end API (Metal itself), through helper toolkit (MetalKit) and helper libraries (Metal Performance Shaders, used for matrix multiplication as well as image filtering and many other things), to high-level frameworks like SpriteKit (2D) and SceneKit (3D), so it's no wonder they are promoting it now.

They want to "be in control" with their pathetic choices and breaking compatibility so that devs have no choice but follow them (in their dreams).

Nobody cares about the stability of private APIs. That is why they are private. A similar division into public and non-public (private/internal/undocumented) APIs is used by many other projects, including Linux.
The decision made by Apple was probably right, because it eliminates the problem of apps incompatibility in the future.

However it still makes no sense to ban MoltenVK, it's absurd in fact. An app could theoretically static link it and not even claim it's using MoltenVK but directly Metal, what would Apple do then?

Nobody bans the application for using MoltenVK, and certainly not Apple. Stop spreading disinformation. The reason was the use of a private/internal/undocumented API, not intended for third-party applications. Even if the application used the Metal API directly, it would still be banned in that case.

See the hypocrisy.

For now, I can only see somebody's stupidity...

So why hasn't Microsoft open-sourced DirectX versions (maybe not 12, since it's the latest) yet?

And why would they do it? I don't see the reason why they would open the Direct3D rendering API or XAudio2 low-level API. As for other components at a higher level, they have already opened them a long time ago:
- DirectXTK: https://github.com/Microsoft/DirectXTK
- DirectXTK12: https://github.com/Microsoft/DirectXTK12
- DirectXMath: https://github.com/Microsoft/DirectXMath
- SHMath: https://github.com/Microsoft/DirectX.../master/SHMath
- DirectXMesh: https://github.com/Microsoft/DirectXMesh
- DirectXTex: https://github.com/Microsoft/DirectXTex
- UVAtlas: https://github.com/Microsoft/UVAtlas
- FX11: https://github.com/Microsoft/FX11
- and even more: https://github.com/Microsoft?q=DirectX

I mean they're not even working on them anymore (DX9, DX11, etc) so they don't gain any benefit of "exposing new features".

DirectX is constantly developed and gets regular updates. These are not just bug fixes, but actually feature updates (e.g. Direct3D 11.4 Version 1709). Moreover, the aforementioned higher-level components (DirectXTK, DirectXMath, DirectXMesh, DirectXTex, UVAtlas, FX11, etc.) are developed independently.
Just tell me one thing. Why do you speak about topics where you have no knowledge?

[horizonbrave]

I guess this ain't gonna be the "Year Of Apple Joining The Linux Foundation" yet

[Weasel]

Exactly, just like in Windows system. No game developer cares about DirectX here, because we have OpenGL, which is a multi-platform, open source standard. Oh, wait. It does not work as you expected.

I don't understand a single thing you're talking about here, care to elaborate? I was asking a rhetorical question, it was sarcasm. They don't like to use open stuff and want to promote their own crap to enforce lock-in later.

It's not like Microsoft trashed OpenGL in the past and tried to kill it by saying how it's subpar (reminds me of ARM's "facts" against RISC-V), nah... oh wait, it did happen. If they didn't care why would they do it?

Moreover, you stayed in the 90s (maybe even earlier), when everyone created their own game engine. This situation no longer exists. Nowadays, developers in most cases choose a third-party game engine, based on their needs:
1. For very specific games you can use dedicated engines, for example: Twine + SugarCube (story format) for text-based novels (interactive, nonlinear stories), RenPy for Visual Novels, RPG Maker for 2D RPGs, PaC-DK (The Point & Click Development Kit) for point-and-click adventure games, etc.
2. For simple 2D games you can use a relatively simple 2D engine, for example Cocos2D or Construct 2 + Tiled (level editor).
3. For medium-range games, you can use Unity 3D + Fabric/FMOD/Wwise (sound engine) + PlayMaker (visual scripting).
4. For more demanding titles you can choose Unreal Engine 4 or maybe even CryEngine (less popular, with worse documentation, generally harder to use).
Most mobile game developers usually go by route 2 or 3. Of course, developing your own engine for really demanding titles still makes sense (e.g. Rockstar Advanced Game Engine, Frostbite or AnvilNext), but it's a niche, especially on mobile market.

The majority of Unreal Engine games have no OpenGL or Vulkan renderer.

There's more Direct3D 12 games out there than Vulkan games (Windows or not). How is this possible, since devs use those engines?!???

There's **FAR** more Direct3D-based games than OpenGL (or Vulkan).

I'm guessing you live in dreamland, so no point arguing further.

(also Unity is crappy and should be left out but that's my opinion I didn't bring it up)

DirectX is constantly developed and gets regular updates. These are not just bug fixes, but actually feature updates (e.g. Direct3D 11.4 Version 1709). Moreover, the aforementioned higher-level components (DirectXTK, DirectXMath, DirectXMesh, DirectXTex, UVAtlas, FX11, etc.) are developed independently.
Just tell me one thing. Why do you speak about topics where you have no knowledge?

We're in a topic about Metal and Vulkan, pure 3D graphics APIs and computing, and yet you think I wasn't clearly talking about Direct3D? Wow, that's some next level of straw grasping you have there.

[Luke]

DirectX was successful on the Windows platform not because MS has forbidden the use of other APIs (what never happened), but because it better met the needs of game developers.

Actually, that's because Windows other than Windows RT is not locked to the microsoft store. Thus, MS never had the ABILITY to prohibit a game developer from using another API or for that matter writing a direct bootable game (can you say GRUB invaders?) that bypasses the entire OS. Also, very few Windows computers other than Surface tablets are purchased from Microsoft, and even the thought that MS would consider locking Windows 8 to the store caused Valve to port some of their games to Linux and develop experimental Steam Machines. I am all but certain that was an insurance policy, a way to bypass any such lock-in.

Not only is Windows/Intel not traditionally a walled garden, it's not been the whole garden at all. Some of our "weeds" sold with Windows have never booted it and only run Linux. For all the trouble of "rooting" a Chromebook when you've never had one before and didn't know not to try and set a root password to reset the boot loader for Linux, that's nothing compared to trying to get an iPhone to boot Debian.

Android (iOS's main competitor) is a mixed bag, with some machines locked, some unlocked, but all allow sideloading without having to root the machine first, and all allow disabling Google Play services and the Google Store. Turn all the anti-privacy shit off, remove or disable the Twitter, Youtube, Google, Facebook and similar apps, and you get a machine whose user-facing code is much more controllable than Apple. Still a weak ecosystem of true FOSS/GPL apps compared to the Linux desktop, posssibly due to needing a totally different build environment. Would be damned nice if it was possible to cross-compile Pluma for Android, given how few decent text editors are in F-droid.

Apple can ban a game from their store, but you can ban their store (and their overpriced phone) from your pocket.