asm/unaligned.h is always an include of asm-generic/unaligned.h;
might as well move that thing to linux/unaligned.h and include
that - there's nothing arch-specific in that header.
auto-generated by the following:
for i in `git grep -l -w asm/unaligned.h`; do
sed -i -e "s/asm\/unaligned.h/linux\/unaligned.h/" $i
done
for i in `git grep -l -w asm-generic/unaligned.h`; do
sed -i -e "s/asm-generic\/unaligned.h/linux\/unaligned.h/" $i
done
git mv include/asm-generic/unaligned.h include/linux/unaligned.h
git mv tools/include/asm-generic/unaligned.h tools/include/linux/unaligned.h
sed -i -e "/unaligned.h/d" include/asm-generic/Kbuild
sed -i -e "s/__ASM_GENERIC/__LINUX/" include/linux/unaligned.h tools/include/linux/unaligned.h
Commit 9744fec95f ("crypto: inside-secure - remove request list to
improve performance") declar this but never implemented.
Signed-off-by: Yue Haibing <yuehaibing@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Macro dma_map_sg() may return 0 on error. This patch enables
checks in case of the macro failure and ensures unmapping of
previously mapped buffers with dma_unmap_sg().
Found by Linux Verification Center (linuxtesting.org) with static
analysis tool SVACE.
Fixes: 49186a7d9e ("crypto: inside_secure - Avoid dma map if size is zero")
Signed-off-by: Nikita Zhandarovich <n.zhandarovich@fintech.ru>
Reviewed-by: Antoine Tenart <atenart@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The .remove() callback for a platform driver returns an int which makes
many driver authors wrongly assume it's possible to do error handling by
returning an error code. However the value returned is ignored (apart
from emitting a warning) and this typically results in resource leaks.
To improve here there is a quest to make the remove callback return
void. In the first step of this quest all drivers are converted to
.remove_new(), which already returns void. Eventually after all drivers
are converted, .remove_new() will be renamed to .remove().
Trivially convert this driver from always returning zero in the remove
callback to the void returning variant.
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
At the moment if there is no firmware available for the safexcel driver
it will fail to load with a cryptic:
crypto-safexcel f2800000.crypto: TRC init: 15360d,80a (48r,256h)
crypto-safexcel f2800000.crypto: HW init failed (-2)
Raise the logging level of the firmware load failure to err rather than
dbg so that it's obvious what the reason for the HW init failure is.
Signed-off-by: Jonathan McDowell <noodles@earth.li>
Reviewed-by: Antoine Tenart <atenart@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This driver has been implicitly relying on kmalloc alignment
to be sufficient for DMA. This may no longer be the case with
upcoming arm64 changes.
This patch changes it to explicitly request DMA alignment from
the Crypto API.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This is to add MaxLinear platform into compatible id.
Firmware endianness option is added since MaxLinear
firmware is in little endian format.
Signed-off-by: Peter Harliman Liem <pliem@maxlinear.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This is to add fw_little_endian option, which can
be used for platform which firmware is using little-endian
(instead of big-endian).
Signed-off-by: Peter Harliman Liem <pliem@maxlinear.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Currently platform data is assigned directly to
version string(instead of struct). To make it more
scalable, we move it to use data struct instead.
This allows customization for individual platforms other
than version string.
Signed-off-by: Peter Harliman Liem <pliem@maxlinear.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Commit 363a90c2d5 ("crypto: safexcel/aes - switch to
library version of key expansion routine") removed
CRYPTO_AES in the config. However, some portions of codes
still rely on generic AES cipher (e.g. refer to
safexcel_aead_gcm_cra_init(), safexcel_xcbcmac_cra_init()).
This causes transform allocation failure for those algos,
if CRYPTO_AES is not manually enabled.
To resolve that, we replace all existing AES cipher
dependent codes with their AES library counterpart.
Fixes: 363a90c2d5 ("crypto: safexcel/aes - switch to library version of key expansion routine")
Signed-off-by: Peter Harliman Liem <pliem@maxlinear.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
From commit d03c544192 ("dma-mapping: disallow .map_sg
operations from returning zero on error"), dma_map_sg()
produces warning if size is 0. This results in visible
warnings if crypto length is zero.
To avoid that, we avoid calling dma_map_sg if size is zero.
Signed-off-by: Peter Harliman Liem <pliem@maxlinear.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The use of swab() is causing failures in 64-bit arch, as it
translates to __swab64() instead of the intended __swab32().
It eventually causes wrong results in xcbcmac & cmac algo.
Fixes: 78cf1c8bfc ("crypto: inside-secure - Move ipad/opad into safexcel_context")
Signed-off-by: Peter Harliman Liem <pliem@maxlinear.com>
Acked-by: Antoine Tenart <atenart@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Without MODULE_DEVICE_TABLE, crypto_safexcel.ko module is not automatically
loaded on platforms where inside-secure crypto HW is specified in device
tree (e.g. Armada 3720). So add missing MODULE_DEVICE_TABLE for of.
Fixes: 1b44c5a60c ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Signed-off-by: Pali Rohár <pali@kernel.org>
Acked-by: Marek Behún <kabel@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When mixing bit-field and none bit-filed in packed struct the
none bit-field starts at a distinct memory location, thus adding
an additional byte to the overall structure which is used in
memory zero-ing and other configuration calculations.
Fix this by removing the none bit-field that has a following
bit-field.
Signed-off-by: Ofer Heifetz <oferh@marvell.com>
Acked-by: Antoine Tenart <atenart@kernel.org>
Acked-by: Antoine Tenart <atenart@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The safexcel module loads firmware so add MODULE_FIRMWARE macros to
provide that information via modinfo.
Signed-off-by: Juerg Haefliger <juergh@protonmail.com>
Acked-by: Antoine Tenart <atenart@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Remove dev_err() messages after platform_get_irq*() failures.
drivers/crypto/inside-secure/safexcel.c: line 1161 is redundant
because platform_get_irq() already prints an error
Generated by: scripts/coccinelle/api/platform_get_irq.cocci
Signed-off-by: Tian Tao <tiantao6@hisilicon.com>
Acked-by: Antoine Tenart <atenart@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The cipher routines in the crypto API are mostly intended for templates
implementing skcipher modes generically in software, and shouldn't be
used outside of the crypto subsystem. So move the prototypes and all
related definitions to a new header file under include/crypto/internal.
Also, let's use the new module namespace feature to move the symbol
exports into a new namespace CRYPTO_INTERNAL.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Acked-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Currently <crypto/sha.h> contains declarations for both SHA-1 and SHA-2,
and <crypto/sha3.h> contains declarations for SHA-3.
This organization is inconsistent, but more importantly SHA-1 is no
longer considered to be cryptographically secure. So to the extent
possible, SHA-1 shouldn't be grouped together with any of the other SHA
versions, and usage of it should be phased out.
Therefore, split <crypto/sha.h> into two headers <crypto/sha1.h> and
<crypto/sha2.h>, and make everyone explicitly specify whether they want
the declarations for SHA-1, SHA-2, or both.
This avoids making the SHA-1 declarations visible to files that don't
want anything to do with SHA-1. It also prepares for potentially moving
sha1.h into a new insecure/ or dangerous/ directory.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Acked-by: Jason A. Donenfeld <Jason@zx2c4.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
An incorrect sizeof() is being used, sizeof(priv->ring[i].rdr_req) is
not correct, it should be sizeof(*priv->ring[i].rdr_req). Note that
since the size of ** is the same size as * this is not causing any
issues.
Addresses-Coverity: ("Sizeof not portable (SIZEOF_MISMATCH)")
Fixes: 9744fec95f ("crypto: inside-secure - remove request list to improve performance")
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Acked-by: Antoine Tenart <atenart@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The code in the current implementation of safexcel_hmac_alg_setkey
can be reused by safexcel_cipher. This patch does just that by
renaming the previous safexcel_hmac_setkey to __safexcel_hmac_setkey.
The now-shared safexcel_hmac_alg_setkey becomes safexcel_hmac_setkey
and a new safexcel_hmac_alg_setkey has been added for use by ahash
transforms.
As a result safexcel_aead_setkey's stack frame has been reduced by
about half in size, or about 512 bytes.
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
As both safexcel_ahash_ctx and safexcel_cipher_ctx contain ipad
and opad buffers this patch moves them into the common struct
safexcel_context. It also adds a union so that they can be accessed
in the appropriate endian without crazy casts.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch moves the priv pointer into struct safexcel_context
because both structs that extend safexcel_context have that pointer
as well.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds support for EIP197 instances that include the output
classifier (OCE) option, as used by one of our biggest customers.
The OCE normally requires initialization and dedicated firmware, but
for the simple operations supported by this driver, we just bypass it
completely for now (using what is formally a debug feature).
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@rambus.com>
Acked-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
On systems with coherence issues, packet processed could succeed while
it should have failed, e.g. because of an authentication fail.
This is because the driver would read stale status information that had
all error bits initialised to zero = no error.
Since this is potential a security risk, we want to prevent it from being
a possibility at all. So initialize all error bits to error state, so
that reading stale status information will always result in errors.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@rambus.com>
Acked-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use kfree_sensitive() instead of open-coding it.
Signed-off-by: Denis Efremov <efremov@linux.com>
Acked-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Balance the irqs of the inside secure driver over all
available cpus.
Currently all interrupts are handled by the first CPU.
From my testing with IPSec AES-GCM 256
on my MCbin with 4 Cores I get a 50% speed increase:
Before the patch: 99.73 Kpps
With the patch: 151.25 Kpps
Signed-off-by: Sven Auhagen <sven.auhagen@voleatech.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The CRYPTO_TFM_RES_* flags were apparently meant as a way to make the
->setkey() functions provide more information about errors. But these
flags weren't actually being used or tested, and in many cases they
weren't being set correctly anyway. So they've now been removed.
Also, if someone ever actually needs to start better distinguishing
->setkey() errors (which is somewhat unlikely, as this has been unneeded
for a long time), we'd be much better off just defining different return
values, like -EINVAL if the key is invalid for the algorithm vs.
-EKEYREJECTED if the key was rejected by a policy like "no weak keys".
That would be much simpler, less error-prone, and easier to test.
So just remove CRYPTO_TFM_RES_MASK and all the unneeded logic that
propagates these flags around.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The CRYPTO_TFM_RES_BAD_KEY_LEN flag was apparently meant as a way to
make the ->setkey() functions provide more information about errors.
However, no one actually checks for this flag, which makes it pointless.
Also, many algorithms fail to set this flag when given a bad length key.
Reviewing just the generic implementations, this is the case for
aes-fixed-time, cbcmac, echainiv, nhpoly1305, pcrypt, rfc3686, rfc4309,
rfc7539, rfc7539esp, salsa20, seqiv, and xcbc. But there are probably
many more in arch/*/crypto/ and drivers/crypto/.
Some algorithms can even set this flag when the key is the correct
length. For example, authenc and authencesn set it when the key payload
is malformed in any way (not just a bad length), the atmel-sha and ccree
drivers can set it if a memory allocation fails, and the chelsio driver
sets it for bad auth tag lengths, not just bad key lengths.
So even if someone actually wanted to start checking this flag (which
seems unlikely, since it's been unused for a long time), there would be
a lot of work needed to get it working correctly. But it would probably
be much better to go back to the drawing board and just define different
return values, like -EINVAL if the key is invalid for the algorithm vs.
-EKEYREJECTED if the key was rejected by a policy like "no weak keys".
That would be much simpler, less error-prone, and easier to test.
So just remove this flag.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Horia Geantă <horia.geanta@nxp.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch fixes another hang case on the EIP97 caused by sending
invalidation tokens to the hardware when doing basic (3)DES ECB/CBC
operations. Invalidation tokens are an EIP197 feature and needed nor
supported by the EIP97. So they should not be sent for that device.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@rambus.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The EIP97 hardware cannot handle zero length input data and will (usually)
hang when presented with this anyway. This patch converts any zero length
input to a 1 byte dummy input to prevent this hanging.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@rambus.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Due to the additions of support for modes like AES-CCM and AES-GCM, which
require large command tokens, the size of the descriptor has grown such that
it now does not fit into the descriptor cache of a standard EIP97 anymore.
This means that the driver no longer works on the Marvell Armada 3700LP chip
(as used on e.g. Espressobin) that it has always supported.
Additionally, performance on EIP197's like Marvell A8K may also degrade
due to being able to fit less descriptors in the on-chip cache.
Putting these tokens into the descriptor was really a hack and not how the
design was supposed to be used - resource allocation did not account for it.
So what this patch does, is move the command token out of the descriptor.
To avoid having to allocate buffers on the fly for these command tokens,
they are stuffed in a "shadow ring", which is a circular buffer of fixed
size blocks that runs in lock-step with the descriptor ring. i.e. there is
one token block per descriptor. The descriptor ring itself is then pre-
populated with the pointers to these token blocks so these do not need to
be filled in when building the descriptors later.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@rambus.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Fixes coccicheck warning:
drivers/crypto/inside-secure/safexcel_cipher.c:2534:1-3: WARNING: PTR_ERR_OR_ZERO can be used
Reported-by: Hulk Robot <hulkci@huawei.com>
Signed-off-by: zhengbin <zhengbin13@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Fixed 2 copy-paste mistakes in the commit mentioned below that caused
authenc w/ (3)DES to consistently fail on Macchiatobin (but strangely
work fine on x86+FPGA??).
Now fully tested on both platforms.
Fixes: 13a1bb93f7 ("crypto: inside-secure - Fixed warnings...")
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Fixed mask used for CFSIZE and RFSIZE fields of HIA_OPTIONS register,
these were all 1 bit too wide. Which caused the probing of a standard
EIP97 to actually hang due to assume way too large descriptor FIFO's.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
safexcel_remove misses disabling priv->reg_clk like what is done when
probe fails.
Add the missed call to fix it.
Signed-off-by: Chuhong Yuan <hslester96@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This fixes a bunch of endianness related sparse warnings reported by the
kbuild test robot as well as Ben Dooks.
Credits for the fix to safexcel.c go to Ben Dooks.
Reported-by: kbuild test robot <lkp@intel.com>
Reported-by: Ben Dooks <ben.dooks@codethink.co.uk>
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
safexcel_pci_remove() is only used locally in the module and not exported,
so added a static function specifier.
This fixes a sparse issue reported by Ben Dooks.
Fixes: 625f269a5a ("crypto: inside-secure - add support for...")
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Always take the zero length hash value for SM3 from the local constant
to avoid a reported build error when SM3 is configured to be a module.
Fixes: 0f2bc13181 ("crypto: inside-secure - Added support for...")
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use PTR_ERR_OR_ZERO rather than if(IS_ERR(...)) + PTR_ERR
Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Acked-by: Antoine Tenart <antoine.tenart@ack.tf>
Acked-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
There is a spelling mistake in a dev_err message. Fix it. Add in missing
newline.
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Acked-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When both PCI and OF are disabled, no drivers are registered, and
we get some unused-function warnings:
drivers/crypto/inside-secure/safexcel.c:1221:13: error: unused function 'safexcel_unregister_algorithms' [-Werror,-Wunused-function]
static void safexcel_unregister_algorithms(struct safexcel_crypto_priv *priv)
drivers/crypto/inside-secure/safexcel.c:1307:12: error: unused function 'safexcel_probe_generic' [-Werror,-Wunused-function]
static int safexcel_probe_generic(void *pdev,
drivers/crypto/inside-secure/safexcel.c:1531:13: error: unused function 'safexcel_hw_reset_rings' [-Werror,-Wunused-function]
static void safexcel_hw_reset_rings(struct safexcel_crypto_priv *priv)
It's better to make the compiler see what is going on and remove
such ifdef checks completely. In case of PCI, this is trivial since
pci_register_driver() is defined to an empty function that makes the
compiler subsequently drop all unused code silently.
The global pcireg_rc/ofreg_rc variables are not actually needed here
since the driver registration does not fail in ways that would make
it helpful.
For CONFIG_OF, an IS_ENABLED() check is still required, since platform
drivers can exist both with and without it.
A little change to linux/pci.h is needed to ensure that
pcim_enable_device() is visible to the driver. Moving the declaration
outside of ifdef would be sufficient here, but for consistency with the
rest of the file, adding an inline helper is probably best.
Fixes: 212ef6f29e ("crypto: inside-secure - Fix unused variable warning when CONFIG_PCI=n")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Bjorn Helgaas <bhelgaas@google.com> # pci.h
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
A previous fixup avoided an unused variable warning but replaced
it with a slightly scarier warning:
drivers/crypto/inside-secure/safexcel.c:1100:6: error: variable 'irq' is used uninitialized whenever 'if' condition is false [-Werror,-Wsometimes-uninitialized]
This is harmless as it is impossible to get into this case, but
the compiler has no way of knowing that. Add an explicit error
handling case to make it obvious to both compilers and humans
reading the source.
Fixes: 212ef6f29e ("crypto: inside-secure - Fix unused variable warning when CONFIG_PCI=n")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds support for the EIP196, which is an EIP197 derivative
that has no classification hardware and a simplified record cache.
The patch has been tested with the eip196b-ie and eip197c-iewxkbc
configurations on the Xilinx VCU118 development board as well as on the
Macchiatobin board (Marvell A8K - EIP197b-ieswx), including the crypto
extra tests.
Note that this patchset applies on top of the earlier submitted
"Add support for eip197f_iewc" series.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The current driver assumes one dedicated ring interrupt controller per
ring. However, some existing EIP(1)97 HW has less ring AIC's than rings.
This patch allows the driver to work with such HW by detecting how many
ring AIC's are present and restricting the number of rings it *uses* by
the number of ring AIC's present. This allows it to at least function.
(optimization for the future: add ring dispatch functionality in the
interrupt service routine such that multiple rings can be supported from
one ring AIC, allowing all rings to be used)
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
