Skip to content

Instantly share code, notes, and snippets.

@panzi
Last active November 14, 2023 07:04
Star You must be signed in to star a gist
Save panzi/6856583 to your computer and use it in GitHub Desktop.
This provides the endian conversion functions form endian.h on Windows, Linux, *BSD, Mac OS X, and QNX. You still need to use -std=gnu99 instead of -std=c99 for gcc. The functions might actually be macros. Functions: htobe16, htole16, be16toh, le16toh, htobe32, htole32, be32toh, le32toh, htobe64, htole64, be64toh, le64toh. License: I hereby put …
// "License": Public Domain
// I, Mathias Panzenböck, place this file hereby into the public domain. Use it at your own risk for whatever you like.
// In case there are jurisdictions that don't support putting things in the public domain you can also consider it to
// be "dual licensed" under the BSD, MIT and Apache licenses, if you want to. This code is trivial anyway. Consider it
// an example on how to get the endian conversion functions on different platforms.
#ifndef PORTABLE_ENDIAN_H__
#define PORTABLE_ENDIAN_H__
#if (defined(_WIN16) || defined(_WIN32) || defined(_WIN64)) && !defined(__WINDOWS__)
# define __WINDOWS__
#endif
#if defined(__linux__) || defined(__CYGWIN__)
# include <endian.h>
#elif defined(__APPLE__)
# include <libkern/OSByteOrder.h>
# define htobe16(x) OSSwapHostToBigInt16(x)
# define htole16(x) OSSwapHostToLittleInt16(x)
# define be16toh(x) OSSwapBigToHostInt16(x)
# define le16toh(x) OSSwapLittleToHostInt16(x)
# define htobe32(x) OSSwapHostToBigInt32(x)
# define htole32(x) OSSwapHostToLittleInt32(x)
# define be32toh(x) OSSwapBigToHostInt32(x)
# define le32toh(x) OSSwapLittleToHostInt32(x)
# define htobe64(x) OSSwapHostToBigInt64(x)
# define htole64(x) OSSwapHostToLittleInt64(x)
# define be64toh(x) OSSwapBigToHostInt64(x)
# define le64toh(x) OSSwapLittleToHostInt64(x)
# define __BYTE_ORDER BYTE_ORDER
# define __BIG_ENDIAN BIG_ENDIAN
# define __LITTLE_ENDIAN LITTLE_ENDIAN
# define __PDP_ENDIAN PDP_ENDIAN
#elif defined(__OpenBSD__)
# include <endian.h>
# define __BYTE_ORDER BYTE_ORDER
# define __BIG_ENDIAN BIG_ENDIAN
# define __LITTLE_ENDIAN LITTLE_ENDIAN
# define __PDP_ENDIAN PDP_ENDIAN
#elif defined(__NetBSD__) || defined(__FreeBSD__) || defined(__DragonFly__)
# include <sys/endian.h>
# define be16toh(x) betoh16(x)
# define le16toh(x) letoh16(x)
# define be32toh(x) betoh32(x)
# define le32toh(x) letoh32(x)
# define be64toh(x) betoh64(x)
# define le64toh(x) letoh64(x)
#elif defined(__WINDOWS__)
# include <winsock2.h>
# ifdef __GNUC__
# include <sys/param.h>
# endif
# if BYTE_ORDER == LITTLE_ENDIAN
# define htobe16(x) htons(x)
# define htole16(x) (x)
# define be16toh(x) ntohs(x)
# define le16toh(x) (x)
# define htobe32(x) htonl(x)
# define htole32(x) (x)
# define be32toh(x) ntohl(x)
# define le32toh(x) (x)
# define htobe64(x) htonll(x)
# define htole64(x) (x)
# define be64toh(x) ntohll(x)
# define le64toh(x) (x)
# elif BYTE_ORDER == BIG_ENDIAN
/* that would be xbox 360 */
# define htobe16(x) (x)
# define htole16(x) __builtin_bswap16(x)
# define be16toh(x) (x)
# define le16toh(x) __builtin_bswap16(x)
# define htobe32(x) (x)
# define htole32(x) __builtin_bswap32(x)
# define be32toh(x) (x)
# define le32toh(x) __builtin_bswap32(x)
# define htobe64(x) (x)
# define htole64(x) __builtin_bswap64(x)
# define be64toh(x) (x)
# define le64toh(x) __builtin_bswap64(x)
# else
# error byte order not supported
# endif
# define __BYTE_ORDER BYTE_ORDER
# define __BIG_ENDIAN BIG_ENDIAN
# define __LITTLE_ENDIAN LITTLE_ENDIAN
# define __PDP_ENDIAN PDP_ENDIAN
#elif defined(__QNXNTO__)
# include <gulliver.h>
# define __LITTLE_ENDIAN 1234
# define __BIG_ENDIAN 4321
# define __PDP_ENDIAN 3412
# if defined(__BIGENDIAN__)
# define __BYTE_ORDER __BIG_ENDIAN
# define htobe16(x) (x)
# define htobe32(x) (x)
# define htobe64(x) (x)
# define htole16(x) ENDIAN_SWAP16(x)
# define htole32(x) ENDIAN_SWAP32(x)
# define htole64(x) ENDIAN_SWAP64(x)
# elif defined(__LITTLEENDIAN__)
# define __BYTE_ORDER __LITTLE_ENDIAN
# define htole16(x) (x)
# define htole32(x) (x)
# define htole64(x) (x)
# define htobe16(x) ENDIAN_SWAP16(x)
# define htobe32(x) ENDIAN_SWAP32(x)
# define htobe64(x) ENDIAN_SWAP64(x)
# else
# error byte order not supported
# endif
# define be16toh(x) ENDIAN_BE16(x)
# define be32toh(x) ENDIAN_BE32(x)
# define be64toh(x) ENDIAN_BE64(x)
# define le16toh(x) ENDIAN_LE16(x)
# define le32toh(x) ENDIAN_LE32(x)
# define le64toh(x) ENDIAN_LE64(x)
#else
# error platform not supported
#endif
#endif
@scunningham
Copy link

Very useful. Any chance you can include the license language in the .h itself?

@markwardle
Copy link

Excellent. Going to use this.

@grodansparadis
Copy link

Perfect, just what I was looking for. Will be used in the VSCP project (http://www.vscp.org)

Copy link

ghost commented Dec 18, 2015

Perfect. Will appear in the Kubux project (http://github.org/KubaKaszycki/kubux)

@blizzard4591
Copy link

I had a problem on a machine running RHEL5, where a GLIBC v2.5 was installed. The macros or functions htobe16 and alike are only present in versions 2.9 and greater.

If you do not mind, please include the following changes:

Replace

#if defined(__linux__) || defined(__CYGWIN__)

#   include <endian.h>

with this:

#if defined(__linux__) || defined(__CYGWIN__)
/* Define necessary macros for the header to expose all fields. */
#   define _BSD_SOURCE 
#   define __USE_BSD
#   define _DEFAULT_SOURCE
#   include <endian.h>
#   include <features.h>
/* See http://linux.die.net/man/3/endian */
#   if !defined(__GLIBC__) || !defined(__GLIBC_MINOR__) || ((__GLIBC__ < 2) || ((__GLIBC__ == 2) && (__GLIBC_MINOR__ < 9))) 
#       include <arpa/inet.h>
#       if defined(__BYTE_ORDER) && (__BYTE_ORDER == __LITTLE_ENDIAN)
#           define htobe16(x) htons(x)
#           define htole16(x) (x)
#           define be16toh(x) ntohs(x)
#           define le16toh(x) (x)

#           define htobe32(x) htonl(x)
#           define htole32(x) (x)
#           define be32toh(x) ntohl(x)
#           define le32toh(x) (x)

#           define htobe64(x) (((uint64_t)htonl(((uint32_t)(((uint64_t)(x)) >> 32)))) | (((uint64_t)htonl(((uint32_t)(x)))) << 32))
#           define htole64(x) (x)
#           define be64toh(x) (((uint64_t)ntohl(((uint32_t)(((uint64_t)(x)) >> 32)))) | (((uint64_t)ntohl(((uint32_t)(x)))) << 32))
#           define le64toh(x) (x)
#       elif defined(__BYTE_ORDER) && (__BYTE_ORDER == __BIG_ENDIAN)
#           define htobe16(x) (x)
#           define htole16(x) ((((((uint16_t)(x)) >> 8))|((((uint16_t)(x)) << 8)))
#           define be16toh(x) (x)
#           define le16toh(x) ((((((uint16_t)(x)) >> 8))|((((uint16_t)(x)) << 8)))

#           define htobe32(x) (x)
#           define htole32(x) (((uint32_t)htole16(((uint16_t)(((uint32_t)(x)) >> 16)))) | (((uint32_t)htole16(((uint16_t)(x)))) << 16))
#           define be32toh(x) (x)
#           define le32toh(x) (((uint32_t)le16toh(((uint16_t)(((uint32_t)(x)) >> 16)))) | (((uint32_t)le16toh(((uint16_t)(x)))) << 16))

#           define htobe64(x) (x)
#           define htole64(x) (((uint64_t)htole32(((uint32_t)(((uint64_t)(x)) >> 32)))) | (((uint64_t)htole32(((uint32_t)(x)))) << 32))
#           define be64toh(x) (x)
#           define le64toh(x) (((uint64_t)le32toh(((uint32_t)(((uint64_t)(x)) >> 32)))) | (((uint64_t)le32toh(((uint32_t)(x)))) << 32))
#       else
#           error Byte Order not supported or not defined.
#       endif
#   endif

@blizzard4591
Copy link

There is a second error: On Windows you include "sys/param.h", which does not exist (maybe on Cygwin, but not with MSVC).

For people using a CMake based approach, I would like to point you to my project: https://github.com/blizzard4591/cmake-portable-endian

It is loosely based on this, but also works if there is no system-based endian-switching support whatsoever.

@vi
Copy link

vi commented Jan 16, 2016

Fails for Android NDK: error: 'be32toh' was not declared in this scope

@n1tehawk
Copy link

n1tehawk commented May 3, 2016

Nice work, thank you! Happily adopted by the sunxi-tools project. - Regards, NiteHawk

@Rahul11111
Copy link

Excellent work. Thank you!

@baz1
Copy link

baz1 commented Aug 21, 2016

Seems very useful. I suggest some hand-written function instead of an error for non-recognized platforms ; less efficiency is always better than a code that does not compile for some reason.

@n1tehawk
Copy link

n1tehawk commented Oct 5, 2016

@panzi
linux-sunxi/sunxi-tools#59 seems to indicate that FreeBSD should be treated like OpenBSD, instead of being the same as NetBSD and DragonFly...

Regards, NiteHawk

@n1tehawk
Copy link

Side note: Using the winsock2 functions for the Windows platform (when BYTE_ORDER == LITTLE_ENDIAN) has the potential drawback of requiring applications to link against the corresponding library (-lws2_32).

@rhdunn
Copy link

rhdunn commented Dec 7, 2016

Nice work. I am using it in espeak-ng as a compatibility shim for endian.h.

@pcordes
Copy link

pcordes commented Apr 26, 2017

According to this SO post and comments here, ntohl doesn't inline on Windows. It's a DLL call instead of a single asm instruction...

Since there's already an #ifdef branch explicitly for LITTLE_ENDIAN Windows, I'd suggest changing avoiding ntohl there.

Pkmx's fork of this gist has that change, using _byteswap_ulong(x) instead of ntohl for MSVC, and __builtin_bswap32(x) for GNU C (gcc, clang, icc).

@zhengqiangmemeda
Copy link

@blizzard4591 ,How to solve the problem of lacking of "sys/param.h" which does not exis on Windows you? I would appreciate it if you can give me some guides

@denniswon
Copy link

@zhengqianman did you resolve sys/param.h issue for windows visual studio? If you did, could you give me some tips on resolve it?

@sjlongland
Copy link

Newlib toolchain users (other than Cygwin), you might want to have a look here:

https://gist.github.com/sjlongland/10642bf86d60bf3d9af04cf504f569da/revisions#diff-a5346b4ffac3d9b1da20a7d99957566f

@ao2
Copy link

ao2 commented Mar 7, 2018

Hi,

the following change adds support for GNU Hurd, and may fix build failures in case portable_endian.h is used in software packaged for Debian, as Hurd is one of the official Debian build targets.

diff --git a/portable_endian.h b/portable_endian.h
index 99845d0..1f9f490 100644
--- a/portable_endian.h
+++ b/portable_endian.h
@@ -13,7 +13,7 @@

 #endif

-#if defined(__linux__) || defined(__CYGWIN__)
+#if defined(__linux__) || defined(__CYGWIN__) || defined(__GNU__)

 #      include <endian.h>

Ciao,
Antonio

@Kaiepi
Copy link

Kaiepi commented Mar 16, 2018

The OpenBSD code should look like this:

#elif defined(__OpenBSD__)

#        include <endian.h>

#        define __BYTE_ORDER    BYTE_ORDER
#        define __BIG_ENDIAN    BIG_ENDIAN
#        define __LITTLE_ENDIAN LITTLE_ENDIAN
#        define __POP_ENDIAN    POP_ENDIAN

Instead of this:

#elif defined(__OpenBSD__)

#	include <sys/endian.h>

@AlexanderStohr
Copy link

#elif defined(__WINDOWS__)

#	include <winsock2.h>
#	include <sys/param.h>

"sys/param.h is a unix/linux header-file, and you shouldn't expect to find that in your Windows system." (unless you are using a GNU compiler.)
https://stackoverflow.com/questions/18333651/vs2010-can-not-open-include-file-sys-param-h

@AlexanderStohr
Copy link

to me it looks like this patch from 2014 adressed it already on the source tree of someone else:
PDAL/PDAL@c432437

(no idea who invented this first and who branched that "idea" from whom - at least the upstream connection seemingly is broken nowadays)

@AlexanderStohr
Copy link

thinking again - your code might work perfectly for GNU C on the windows platform.
guess what - people are trying out to compile it with MSVC - and its just not documented that you were not targeting that compiler.

@proski
Copy link

proski commented Feb 20, 2019

Defining __WINDOWS__ is not good. #ifdef __WINDOWS__ can work differently in files that include and don't include endian.h, which would be surprising. It's better to define symbols that communicate their limited scope, e.g. __ENDIAN_WINDOWS.
The use of __builtin_bswapNN whould be limited to the compiles that define it (GCC and Clang, both define __GNUC__).

@xyz1001
Copy link

xyz1001 commented Apr 20, 2021

This header is useful. I want to create a conan package. Counld you please create a repo for this gist to support a better version control?

@panzi
Copy link
Author

panzi commented Apr 20, 2021

I haven't updated this in 6 years, I think it's save to say there won't be any new version of this. I'm not maintaining it. 😄
Just copy the file/code wherever you need it. I really don't see the benefit of a full repo for that. All that would do is people writing issues that I don't feel like fixing. A gist makes clear its just an example on how one can do something like this.

@pablorcum
Copy link

There is currently no support for QNX Neutrino. You can add it easily as shown below:

#elif defined(__QNXNTO__)

#	include <gulliver.h>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment