public inbox for drm-ai-reviews@public-inbox.freedesktop.org
 help / color / mirror / Atom feed
* [PATCH v2 00/10] fbcon,fonts: Refactor framebuffer console rotation
@ 2026-04-07  9:23 Thomas Zimmermann
  2026-04-07  9:23 ` [PATCH v2 01/10] fbcon: Avoid OOB font access if console rotation fails Thomas Zimmermann
                   ` (10 more replies)
  0 siblings, 11 replies; 23+ messages in thread
From: Thomas Zimmermann @ 2026-04-07  9:23 UTC (permalink / raw)
  To: deller, gregkh, jirislaby, geert, simona, sam
  Cc: linux-fbdev, dri-devel, linux-kernel, linux-serial,
	Thomas Zimmermann

Refactor the framebuffer console rotation into individual components
for glyphs, fonts and the overall fbcon state. Right now this is mixed
up in fbcon_rotate.{c,h}. Also build cursor rotation on top of the new
interfaces.

Start with an OOB fix in patch 1. If buffer allocation fails, fbcon
currently uses a too-small glyph buffer for output. Avoid that.

Patches 2 to 4 make a number of small improvements to the font library
and its callers.

Patches 5 to 8 refactor the font rotation. Fbcon rotation rotates each
individual glyph in a font buffer and uses the rotated buffer's glyphs
for output. The result looks like the console buffer has been rotated
as a whole. Split this into helpers that rotate individual glyphs and
a helper that rotates the font buffer of these. Then reimplement fbcon
rotation on top. Document the public font helpers.

Patch 9 rebuilds cursor rotation on top of the new glyph helpers. The
fbcon cursor is itself a glyph that has to be rotated in sync with the
font.

Patch 10 moves all state of fbcon rotation into a single place and makes
it a build-time conditional.

Tested with fbcon under bochs on Qemu.

Built upon the fbcon changes at [1].

v2:
- fix sparse truncated-bits warning
- improve font sorting in Makefile
- fix typos

[1] https://lore.kernel.org/linux-fbdev/20260309141723.137364-1-tzimmermann@suse.de/

Thomas Zimmermann (10):
  fbcon: Avoid OOB font access if console rotation fails
  vt: Implement helpers for struct vc_font in source file
  lib/fonts: Provide helpers for calculating glyph pitch and size
  lib/fonts: Clean up Makefile
  lib/fonts: Implement glyph rotation
  lib/fonts: Refactor glyph-pattern helpers
  lib/fonts: Refactor glyph-rotation helpers
  lib/fonts: Implement font rotation
  fbcon: Fill cursor mask in helper function
  fbcon: Put font-rotation state into separate struct

 drivers/tty/vt/vt.c                     |  34 +++
 drivers/video/fbdev/core/bitblit.c      |  35 +--
 drivers/video/fbdev/core/fbcon.c        |  48 ++++-
 drivers/video/fbdev/core/fbcon.h        |  14 +-
 drivers/video/fbdev/core/fbcon_ccw.c    |  70 ++----
 drivers/video/fbdev/core/fbcon_cw.c     |  70 ++----
 drivers/video/fbdev/core/fbcon_rotate.c |  88 ++------
 drivers/video/fbdev/core/fbcon_rotate.h |  71 ------
 drivers/video/fbdev/core/fbcon_ud.c     |  67 ++----
 include/linux/console_struct.h          |  30 +--
 include/linux/font.h                    |  51 +++++
 lib/fonts/Makefile                      |  36 ++--
 lib/fonts/font_rotate.c                 | 275 ++++++++++++++++++++++++
 lib/fonts/fonts.c                       |   2 +-
 14 files changed, 525 insertions(+), 366 deletions(-)
 create mode 100644 lib/fonts/font_rotate.c

-- 
2.53.0


^ permalink raw reply	[flat|nested] 23+ messages in thread

* [PATCH v2 01/10] fbcon: Avoid OOB font access if console rotation fails
  2026-04-07  9:23 [PATCH v2 00/10] fbcon,fonts: Refactor framebuffer console rotation Thomas Zimmermann
@ 2026-04-07  9:23 ` Thomas Zimmermann
  2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
  2026-04-07  9:23 ` [PATCH v2 02/10] vt: Implement helpers for struct vc_font in source file Thomas Zimmermann
                   ` (9 subsequent siblings)
  10 siblings, 1 reply; 23+ messages in thread
From: Thomas Zimmermann @ 2026-04-07  9:23 UTC (permalink / raw)
  To: deller, gregkh, jirislaby, geert, simona, sam
  Cc: linux-fbdev, dri-devel, linux-kernel, linux-serial,
	Thomas Zimmermann, stable

Clear the font buffer if the reallocation during console rotation fails
in fbcon_rotate_font(). The putcs implementations for the rotated buffer
will return early in this case. See [1] for an example.

Currently, fbcon_rotate_font() keeps the old buffer, which is too small
for the rotated font. Printing to the rotated console with a high-enough
character code will overflow the font buffer.

v2:
- fix typos in commit message

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Fixes: 6cc50e1c5b57 ("[PATCH] fbcon: Console Rotation - Add support to rotate font bitmap")
Cc: <stable@vger.kernel.org> # v2.6.15+
Link: https://elixir.bootlin.com/linux/v6.19/source/drivers/video/fbdev/core/fbcon_ccw.c#L144 # [1]
---
 drivers/video/fbdev/core/fbcon_rotate.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/video/fbdev/core/fbcon_rotate.c b/drivers/video/fbdev/core/fbcon_rotate.c
index 1562a8f20b4f..5348f6c6f57c 100644
--- a/drivers/video/fbdev/core/fbcon_rotate.c
+++ b/drivers/video/fbdev/core/fbcon_rotate.c
@@ -46,6 +46,10 @@ int fbcon_rotate_font(struct fb_info *info, struct vc_data *vc)
 		info->fbops->fb_sync(info);
 
 	if (par->fd_size < d_cellsize * len) {
+		kfree(par->fontbuffer);
+		par->fontbuffer = NULL;
+		par->fd_size = 0;
+
 		dst = kmalloc_array(len, d_cellsize, GFP_KERNEL);
 
 		if (dst == NULL) {
@@ -54,7 +58,6 @@ int fbcon_rotate_font(struct fb_info *info, struct vc_data *vc)
 		}
 
 		par->fd_size = d_cellsize * len;
-		kfree(par->fontbuffer);
 		par->fontbuffer = dst;
 	}
 
-- 
2.53.0


^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH v2 02/10] vt: Implement helpers for struct vc_font in source file
  2026-04-07  9:23 [PATCH v2 00/10] fbcon,fonts: Refactor framebuffer console rotation Thomas Zimmermann
  2026-04-07  9:23 ` [PATCH v2 01/10] fbcon: Avoid OOB font access if console rotation fails Thomas Zimmermann
@ 2026-04-07  9:23 ` Thomas Zimmermann
  2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
  2026-04-07  9:23 ` [PATCH v2 03/10] lib/fonts: Provide helpers for calculating glyph pitch and size Thomas Zimmermann
                   ` (8 subsequent siblings)
  10 siblings, 1 reply; 23+ messages in thread
From: Thomas Zimmermann @ 2026-04-07  9:23 UTC (permalink / raw)
  To: deller, gregkh, jirislaby, geert, simona, sam
  Cc: linux-fbdev, dri-devel, linux-kernel, linux-serial,
	Thomas Zimmermann

Move the helpers vc_font_pitch() and vc_font_size() from the VT
header file into source file. They are not called very often, so
there's no benefit in keeping them in the headers. Also avoids
including <linux/math.h> from the header.

v2:
- fix typo in commit description

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
 drivers/tty/vt/vt.c            | 35 ++++++++++++++++++++++++++++++++++
 include/linux/console_struct.h | 30 ++---------------------------
 2 files changed, 37 insertions(+), 28 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index e2df99e3d458..3d89d30c9596 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -71,6 +71,7 @@
  * by Adam Tla/lka <atlka@pg.gda.pl>, Aug 2006
  */
 
+#include <linux/math.h>
 #include <linux/module.h>
 #include <linux/types.h>
 #include <linux/sched/signal.h>
@@ -230,6 +231,40 @@ enum {
 	blank_vesa_wait,
 };
 
+/*
+ * struct vc_font
+ */
+
+/**
+ * vc_font_pitch - Calculates the number of bytes between two adjacent scanlines
+ * @font: The VC font
+ *
+ * Returns:
+ * The number of bytes between two adjacent scanlines in the font data
+ */
+unsigned int vc_font_pitch(const struct vc_font *font)
+{
+	return DIV_ROUND_UP(font->width, 8);
+}
+EXPORT_SYMBOL_GPL(vc_font_pitch);
+
+/**
+ * vc_font_size - Calculates the size of the font data in bytes
+ * @font: The VC font
+ *
+ * vc_font_size() calculates the number of bytes of font data in the
+ * font specified by @font. The function calculates the size from the
+ * font parameters.
+ *
+ * Returns:
+ * The size of the font data in bytes.
+ */
+unsigned int vc_font_size(const struct vc_font *font)
+{
+	return font->height * vc_font_pitch(font) * font->charcount;
+}
+EXPORT_SYMBOL_GPL(vc_font_size);
+
 /*
  * /sys/class/tty/tty0/
  *
diff --git a/include/linux/console_struct.h b/include/linux/console_struct.h
index 6ce498b31855..fe915afdece5 100644
--- a/include/linux/console_struct.h
+++ b/include/linux/console_struct.h
@@ -13,7 +13,6 @@
 #ifndef _LINUX_CONSOLE_STRUCT_H
 #define _LINUX_CONSOLE_STRUCT_H
 
-#include <linux/math.h>
 #include <linux/vt.h>
 #include <linux/wait.h>
 #include <linux/workqueue.h>
@@ -83,33 +82,8 @@ struct vc_font {
 	const unsigned char *data;
 };
 
-/**
- * vc_font_pitch - Calculates the number of bytes between two adjacent scanlines
- * @font: The VC font
- *
- * Returns:
- * The number of bytes between two adjacent scanlines in the font data
- */
-static inline unsigned int vc_font_pitch(const struct vc_font *font)
-{
-	return DIV_ROUND_UP(font->width, 8);
-}
-
-/**
- * vc_font_size - Calculates the size of the font data in bytes
- * @font: The VC font
- *
- * vc_font_size() calculates the number of bytes of font data in the
- * font specified by @font. The function calculates the size from the
- * font parameters.
- *
- * Returns:
- * The size of the font data in bytes.
- */
-static inline unsigned int vc_font_size(const struct vc_font *font)
-{
-	return font->height * vc_font_pitch(font) * font->charcount;
-}
+unsigned int vc_font_pitch(const struct vc_font *font);
+unsigned int vc_font_size(const struct vc_font *font);
 
 /*
  * Example: vc_data of a console that was scrolled 3 lines down.
-- 
2.53.0


^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH v2 03/10] lib/fonts: Provide helpers for calculating glyph pitch and size
  2026-04-07  9:23 [PATCH v2 00/10] fbcon,fonts: Refactor framebuffer console rotation Thomas Zimmermann
  2026-04-07  9:23 ` [PATCH v2 01/10] fbcon: Avoid OOB font access if console rotation fails Thomas Zimmermann
  2026-04-07  9:23 ` [PATCH v2 02/10] vt: Implement helpers for struct vc_font in source file Thomas Zimmermann
@ 2026-04-07  9:23 ` Thomas Zimmermann
  2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
  2026-04-07  9:23 ` [PATCH v2 04/10] lib/fonts: Clean up Makefile Thomas Zimmermann
                   ` (7 subsequent siblings)
  10 siblings, 1 reply; 23+ messages in thread
From: Thomas Zimmermann @ 2026-04-07  9:23 UTC (permalink / raw)
  To: deller, gregkh, jirislaby, geert, simona, sam
  Cc: linux-fbdev, dri-devel, linux-kernel, linux-serial,
	Thomas Zimmermann

Implement pitch and size calculation for a single font glyph in the
new helpers font_glyph_pitch() and font_glyph_size(). Replace the
instances where the calculations are open-coded.

Note that in the case of fbcon console rotation, the parameters for
a glyph's width and height might be reversed. This is intentional.

v2:
- fix typos in commit message

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/tty/vt/vt.c                     |  5 ++--
 drivers/video/fbdev/core/fbcon_ccw.c    | 11 +++----
 drivers/video/fbdev/core/fbcon_cw.c     | 11 +++----
 drivers/video/fbdev/core/fbcon_rotate.c |  6 ++--
 drivers/video/fbdev/core/fbcon_ud.c     |  7 +++--
 include/linux/font.h                    | 40 +++++++++++++++++++++++++
 lib/fonts/fonts.c                       |  2 +-
 7 files changed, 61 insertions(+), 21 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 3d89d30c9596..23b9683b52a5 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -71,7 +71,6 @@
  * by Adam Tla/lka <atlka@pg.gda.pl>, Aug 2006
  */
 
-#include <linux/math.h>
 #include <linux/module.h>
 #include <linux/types.h>
 #include <linux/sched/signal.h>
@@ -244,7 +243,7 @@ enum {
  */
 unsigned int vc_font_pitch(const struct vc_font *font)
 {
-	return DIV_ROUND_UP(font->width, 8);
+	return font_glyph_pitch(font->width);
 }
 EXPORT_SYMBOL_GPL(vc_font_pitch);
 
@@ -261,7 +260,7 @@ EXPORT_SYMBOL_GPL(vc_font_pitch);
  */
 unsigned int vc_font_size(const struct vc_font *font)
 {
-	return font->height * vc_font_pitch(font) * font->charcount;
+	return font_glyph_size(font->width, font->height) * font->charcount;
 }
 EXPORT_SYMBOL_GPL(vc_font_size);
 
diff --git a/drivers/video/fbdev/core/fbcon_ccw.c b/drivers/video/fbdev/core/fbcon_ccw.c
index 2f394b5a17f7..96ef449ee6ac 100644
--- a/drivers/video/fbdev/core/fbcon_ccw.c
+++ b/drivers/video/fbdev/core/fbcon_ccw.c
@@ -26,7 +26,7 @@ static void ccw_update_attr(u8 *dst, u8 *src, int attribute,
 				  struct vc_data *vc)
 {
 	int i, j, offset = (vc->vc_font.height < 10) ? 1 : 2;
-	int width = (vc->vc_font.height + 7) >> 3;
+	int width = font_glyph_pitch(vc->vc_font.height);
 	int mod = vc->vc_font.height % 8;
 	u8 c, msk = ~(0xff << offset), msk1 = 0;
 
@@ -101,7 +101,7 @@ static inline void ccw_putcs_aligned(struct vc_data *vc, struct fb_info *info,
 {
 	struct fbcon_par *par = info->fbcon_par;
 	u16 charmask = vc->vc_hi_font_mask ? 0x1ff : 0xff;
-	u32 idx = (vc->vc_font.height + 7) >> 3;
+	u32 idx = font_glyph_pitch(vc->vc_font.height);
 	u8 *src;
 
 	while (cnt--) {
@@ -131,7 +131,7 @@ static void ccw_putcs(struct vc_data *vc, struct fb_info *info,
 {
 	struct fb_image image;
 	struct fbcon_par *par = info->fbcon_par;
-	u32 width = (vc->vc_font.height + 7)/8;
+	u32 width = font_glyph_pitch(vc->vc_font.height);
 	u32 cellsize = width * vc->vc_font.width;
 	u32 maxcnt = info->pixmap.size/cellsize;
 	u32 scan_align = info->pixmap.scan_align - 1;
@@ -223,7 +223,8 @@ static void ccw_cursor(struct vc_data *vc, struct fb_info *info, bool enable,
 	struct fb_cursor cursor;
 	struct fbcon_par *par = info->fbcon_par;
 	unsigned short charmask = vc->vc_hi_font_mask ? 0x1ff : 0xff;
-	int w = (vc->vc_font.height + 7) >> 3, c;
+	int w = font_glyph_pitch(vc->vc_font.height);
+	int c;
 	int y = real_y(par->p, vc->state.y);
 	int attribute, use_sw = vc->vc_cursor_type & CUR_SW;
 	int err = 1, dx, dy;
@@ -297,7 +298,7 @@ static void ccw_cursor(struct vc_data *vc, struct fb_info *info, bool enable,
 		char *tmp, *mask = kmalloc_array(w, vc->vc_font.width,
 						 GFP_ATOMIC);
 		int cur_height, size, i = 0;
-		int width = (vc->vc_font.width + 7)/8;
+		int width = font_glyph_pitch(vc->vc_font.width);
 
 		if (!mask)
 			return;
diff --git a/drivers/video/fbdev/core/fbcon_cw.c b/drivers/video/fbdev/core/fbcon_cw.c
index 3c3ad3471ec4..ea712654edae 100644
--- a/drivers/video/fbdev/core/fbcon_cw.c
+++ b/drivers/video/fbdev/core/fbcon_cw.c
@@ -26,7 +26,7 @@ static void cw_update_attr(u8 *dst, u8 *src, int attribute,
 				  struct vc_data *vc)
 {
 	int i, j, offset = (vc->vc_font.height < 10) ? 1 : 2;
-	int width = (vc->vc_font.height + 7) >> 3;
+	int width = font_glyph_pitch(vc->vc_font.height);
 	u8 c, msk = ~(0xff >> offset);
 
 	for (i = 0; i < vc->vc_font.width; i++) {
@@ -86,7 +86,7 @@ static inline void cw_putcs_aligned(struct vc_data *vc, struct fb_info *info,
 {
 	struct fbcon_par *par = info->fbcon_par;
 	u16 charmask = vc->vc_hi_font_mask ? 0x1ff : 0xff;
-	u32 idx = (vc->vc_font.height + 7) >> 3;
+	u32 idx = font_glyph_pitch(vc->vc_font.height);
 	u8 *src;
 
 	while (cnt--) {
@@ -116,7 +116,7 @@ static void cw_putcs(struct vc_data *vc, struct fb_info *info,
 {
 	struct fb_image image;
 	struct fbcon_par *par = info->fbcon_par;
-	u32 width = (vc->vc_font.height + 7)/8;
+	u32 width = font_glyph_pitch(vc->vc_font.height);
 	u32 cellsize = width * vc->vc_font.width;
 	u32 maxcnt = info->pixmap.size/cellsize;
 	u32 scan_align = info->pixmap.scan_align - 1;
@@ -206,7 +206,8 @@ static void cw_cursor(struct vc_data *vc, struct fb_info *info, bool enable,
 	struct fb_cursor cursor;
 	struct fbcon_par *par = info->fbcon_par;
 	unsigned short charmask = vc->vc_hi_font_mask ? 0x1ff : 0xff;
-	int w = (vc->vc_font.height + 7) >> 3, c;
+	int w = font_glyph_pitch(vc->vc_font.height);
+	int c;
 	int y = real_y(par->p, vc->state.y);
 	int attribute, use_sw = vc->vc_cursor_type & CUR_SW;
 	int err = 1, dx, dy;
@@ -280,7 +281,7 @@ static void cw_cursor(struct vc_data *vc, struct fb_info *info, bool enable,
 		char *tmp, *mask = kmalloc_array(w, vc->vc_font.width,
 						 GFP_ATOMIC);
 		int cur_height, size, i = 0;
-		int width = (vc->vc_font.width + 7)/8;
+		int width = font_glyph_pitch(vc->vc_font.width);
 
 		if (!mask)
 			return;
diff --git a/drivers/video/fbdev/core/fbcon_rotate.c b/drivers/video/fbdev/core/fbcon_rotate.c
index 5348f6c6f57c..18575c5182db 100644
--- a/drivers/video/fbdev/core/fbcon_rotate.c
+++ b/drivers/video/fbdev/core/fbcon_rotate.c
@@ -33,14 +33,12 @@ int fbcon_rotate_font(struct fb_info *info, struct vc_data *vc)
 	src = par->fontdata = vc->vc_font.data;
 	par->cur_rotate = par->p->con_rotate;
 	len = vc->vc_font.charcount;
-	s_cellsize = ((vc->vc_font.width + 7)/8) *
-		vc->vc_font.height;
+	s_cellsize = font_glyph_size(vc->vc_font.width, vc->vc_font.height);
 	d_cellsize = s_cellsize;
 
 	if (par->rotate == FB_ROTATE_CW ||
 	    par->rotate == FB_ROTATE_CCW)
-		d_cellsize = ((vc->vc_font.height + 7)/8) *
-			vc->vc_font.width;
+		d_cellsize = font_glyph_size(vc->vc_font.height, vc->vc_font.width);
 
 	if (info->fbops->fb_sync)
 		info->fbops->fb_sync(info);
diff --git a/drivers/video/fbdev/core/fbcon_ud.c b/drivers/video/fbdev/core/fbcon_ud.c
index 6fc30cad5b19..f7cd89c42b01 100644
--- a/drivers/video/fbdev/core/fbcon_ud.c
+++ b/drivers/video/fbdev/core/fbcon_ud.c
@@ -26,7 +26,7 @@ static void ud_update_attr(u8 *dst, u8 *src, int attribute,
 				  struct vc_data *vc)
 {
 	int i, offset = (vc->vc_font.height < 10) ? 1 : 2;
-	int width = (vc->vc_font.width + 7) >> 3;
+	int width = font_glyph_pitch(vc->vc_font.width);
 	unsigned int cellsize = vc->vc_font.height * width;
 	u8 c;
 
@@ -153,7 +153,7 @@ static void ud_putcs(struct vc_data *vc, struct fb_info *info,
 {
 	struct fb_image image;
 	struct fbcon_par *par = info->fbcon_par;
-	u32 width = (vc->vc_font.width + 7)/8;
+	u32 width = font_glyph_pitch(vc->vc_font.width);
 	u32 cellsize = width * vc->vc_font.height;
 	u32 maxcnt = info->pixmap.size/cellsize;
 	u32 scan_align = info->pixmap.scan_align - 1;
@@ -253,7 +253,8 @@ static void ud_cursor(struct vc_data *vc, struct fb_info *info, bool enable,
 	struct fb_cursor cursor;
 	struct fbcon_par *par = info->fbcon_par;
 	unsigned short charmask = vc->vc_hi_font_mask ? 0x1ff : 0xff;
-	int w = (vc->vc_font.width + 7) >> 3, c;
+	int w = font_glyph_pitch(vc->vc_font.width);
+	int c;
 	int y = real_y(par->p, vc->state.y);
 	int attribute, use_sw = vc->vc_cursor_type & CUR_SW;
 	int err = 1, dx, dy;
diff --git a/include/linux/font.h b/include/linux/font.h
index 5401f07dd6ce..3bd49d914b22 100644
--- a/include/linux/font.h
+++ b/include/linux/font.h
@@ -11,10 +11,50 @@
 #ifndef _VIDEO_FONT_H
 #define _VIDEO_FONT_H
 
+#include <linux/math.h>
 #include <linux/types.h>
 
 struct console_font;
 
+/*
+ * Glyphs
+ */
+
+/**
+ * font_glyph_pitch - Calculates the number of bytes per scanline
+ * @width: The glyph width in bits per scanline
+ *
+ * A glyph's pitch is the number of bytes in a single scanline, rounded
+ * up to the next full byte. The parameter @width receives the number
+ * of visible bits per scanline. For example, if width is 14 bytes per
+ * scanline, the pitch is 2 bytes per scanline. If width is 8 bits per
+ * scanline, the pitch is 1 byte per scanline.
+ *
+ * Returns:
+ * The number of bytes in a single scanline of the glyph
+ */
+static inline unsigned int font_glyph_pitch(unsigned int width)
+{
+	return DIV_ROUND_UP(width, 8);
+}
+
+/**
+ * font_glyph_size - Calculates the number of bytes per glyph
+ * @width: The glyph width in bits per scanline
+ * @vpitch: The number of scanlines in the glyph
+ *
+ * The number of bytes in a glyph depends on the pitch and the number
+ * of scanlines. font_glyph_size automatically calculates the pitch
+ * from the given width. The parameter @vpitch gives the number of
+ * scanlines, which is usually the glyph's height in scanlines. Fonts
+ * coming from user space can sometimes have a different vertical pitch
+ * with empty scanlines between two adjacent glyphs.
+ */
+static inline unsigned int font_glyph_size(unsigned int width, unsigned int vpitch)
+{
+	return font_glyph_pitch(width) * vpitch;
+}
+
 /*
  * font_data_t and helpers
  */
diff --git a/lib/fonts/fonts.c b/lib/fonts/fonts.c
index 5938f542906b..f5d5333450a0 100644
--- a/lib/fonts/fonts.c
+++ b/lib/fonts/fonts.c
@@ -26,7 +26,7 @@
 
 #include "font.h"
 
-#define console_font_pitch(font) DIV_ROUND_UP((font)->width, 8)
+#define console_font_pitch(font) font_glyph_pitch((font)->width)
 
 /*
  * Helpers for font_data_t
-- 
2.53.0


^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH v2 04/10] lib/fonts: Clean up Makefile
  2026-04-07  9:23 [PATCH v2 00/10] fbcon,fonts: Refactor framebuffer console rotation Thomas Zimmermann
                   ` (2 preceding siblings ...)
  2026-04-07  9:23 ` [PATCH v2 03/10] lib/fonts: Provide helpers for calculating glyph pitch and size Thomas Zimmermann
@ 2026-04-07  9:23 ` Thomas Zimmermann
  2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
  2026-04-07  9:23 ` [PATCH v2 05/10] lib/fonts: Implement glyph rotation Thomas Zimmermann
                   ` (6 subsequent siblings)
  10 siblings, 1 reply; 23+ messages in thread
From: Thomas Zimmermann @ 2026-04-07  9:23 UTC (permalink / raw)
  To: deller, gregkh, jirislaby, geert, simona, sam
  Cc: linux-fbdev, dri-devel, linux-kernel, linux-serial,
	Thomas Zimmermann

Simplify the Makefile. Drop font-obj-y and sort the fonts by dictionary
order. Done in preparation for supporting optional font rotation.

v2:
- sort Makefile font entries by Family/Size in ascending order (Geert, Jiri)

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 lib/fonts/Makefile | 35 +++++++++++++++++------------------
 1 file changed, 17 insertions(+), 18 deletions(-)

diff --git a/lib/fonts/Makefile b/lib/fonts/Makefile
index 30a85a4292fa..b176af53d53e 100644
--- a/lib/fonts/Makefile
+++ b/lib/fonts/Makefile
@@ -1,23 +1,22 @@
 # SPDX-License-Identifier: GPL-2.0
 # Font handling
 
-font-objs := fonts.o
+font-y := fonts.o
 
-font-objs-$(CONFIG_FONT_SUN8x16)   += font_sun8x16.o
-font-objs-$(CONFIG_FONT_SUN12x22)  += font_sun12x22.o
-font-objs-$(CONFIG_FONT_8x8)       += font_8x8.o
-font-objs-$(CONFIG_FONT_8x16)      += font_8x16.o
-font-objs-$(CONFIG_FONT_6x11)      += font_6x11.o
-font-objs-$(CONFIG_FONT_7x14)      += font_7x14.o
-font-objs-$(CONFIG_FONT_10x18)     += font_10x18.o
-font-objs-$(CONFIG_FONT_PEARL_8x8) += font_pearl_8x8.o
-font-objs-$(CONFIG_FONT_ACORN_8x8) += font_acorn_8x8.o
-font-objs-$(CONFIG_FONT_MINI_4x6)  += font_mini_4x6.o
-font-objs-$(CONFIG_FONT_6x10)      += font_6x10.o
-font-objs-$(CONFIG_FONT_TER10x18)  += font_ter10x18.o
-font-objs-$(CONFIG_FONT_TER16x32)  += font_ter16x32.o
-font-objs-$(CONFIG_FONT_6x8)       += font_6x8.o
+# Built-in fonts; sorted by Family-Size in ascending order
+font-$(CONFIG_FONT_6x8)       += font_6x8.o
+font-$(CONFIG_FONT_6x10)      += font_6x10.o
+font-$(CONFIG_FONT_6x11)      += font_6x11.o
+font-$(CONFIG_FONT_7x14)      += font_7x14.o
+font-$(CONFIG_FONT_8x8)       += font_8x8.o
+font-$(CONFIG_FONT_8x16)      += font_8x16.o
+font-$(CONFIG_FONT_10x18)     += font_10x18.o
+font-$(CONFIG_FONT_ACORN_8x8) += font_acorn_8x8.o
+font-$(CONFIG_FONT_MINI_4x6)  += font_mini_4x6.o
+font-$(CONFIG_FONT_PEARL_8x8) += font_pearl_8x8.o
+font-$(CONFIG_FONT_SUN8x16)   += font_sun8x16.o
+font-$(CONFIG_FONT_SUN12x22)  += font_sun12x22.o
+font-$(CONFIG_FONT_TER10x18)  += font_ter10x18.o
+font-$(CONFIG_FONT_TER16x32)  += font_ter16x32.o
 
-font-objs += $(font-objs-y)
-
-obj-$(CONFIG_FONT_SUPPORT)         += font.o
+obj-$(CONFIG_FONT_SUPPORT) += font.o
-- 
2.53.0


^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH v2 05/10] lib/fonts: Implement glyph rotation
  2026-04-07  9:23 [PATCH v2 00/10] fbcon,fonts: Refactor framebuffer console rotation Thomas Zimmermann
                   ` (3 preceding siblings ...)
  2026-04-07  9:23 ` [PATCH v2 04/10] lib/fonts: Clean up Makefile Thomas Zimmermann
@ 2026-04-07  9:23 ` Thomas Zimmermann
  2026-04-07 15:48   ` Helge Deller
  2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
  2026-04-07  9:23 ` [PATCH v2 06/10] lib/fonts: Refactor glyph-pattern helpers Thomas Zimmermann
                   ` (5 subsequent siblings)
  10 siblings, 2 replies; 23+ messages in thread
From: Thomas Zimmermann @ 2026-04-07  9:23 UTC (permalink / raw)
  To: deller, gregkh, jirislaby, geert, simona, sam
  Cc: linux-fbdev, dri-devel, linux-kernel, linux-serial,
	Thomas Zimmermann

Move the glyph rotation helpers from fbcon to the font library. Wrap them
behind clean interfaces. Also clear the output memory to zero. Previously,
the implementation relied on the caller to do that.

Go through the fbcon code and callers of the glyph-rotation helpers. In
addition to the font rotation, there's also the cursor code, which uses
the rotation helpers.

The font-rotation relied on a single memset to zero for the whole font.
This is now multiple memsets on each glyph. This will be sorted out when
the font library also implements font rotation.

Building glyph rotation in the font library still depends on
CONFIG_FRAMEBUFFER_CONSOLE_ROTATION=y. If we get more users of the code,
we can still add a dedicated Kconfig symbol to the font library.

No changes have been made to the actual implementation of the rotate_*()
and pattern_*() functions. These will be refactored as separate changes.

v2:
- fix typos

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/video/fbdev/core/fbcon_ccw.c    |   4 +-
 drivers/video/fbdev/core/fbcon_cw.c     |   4 +-
 drivers/video/fbdev/core/fbcon_rotate.c |  12 +-
 drivers/video/fbdev/core/fbcon_rotate.h |  71 -----------
 include/linux/font.h                    |   8 ++
 lib/fonts/Makefile                      |   1 +
 lib/fonts/font_rotate.c                 | 150 ++++++++++++++++++++++++
 7 files changed, 167 insertions(+), 83 deletions(-)
 create mode 100644 lib/fonts/font_rotate.c

diff --git a/drivers/video/fbdev/core/fbcon_ccw.c b/drivers/video/fbdev/core/fbcon_ccw.c
index 96ef449ee6ac..72453a2aaca8 100644
--- a/drivers/video/fbdev/core/fbcon_ccw.c
+++ b/drivers/video/fbdev/core/fbcon_ccw.c
@@ -12,6 +12,7 @@
 #include <linux/slab.h>
 #include <linux/string.h>
 #include <linux/fb.h>
+#include <linux/font.h>
 #include <linux/vt_kern.h>
 #include <linux/console.h>
 #include <asm/types.h>
@@ -344,8 +345,7 @@ static void ccw_cursor(struct vc_data *vc, struct fb_info *info, bool enable,
 		size = cur_height * width;
 		while (size--)
 			tmp[i++] = 0xff;
-		memset(mask, 0, w * vc->vc_font.width);
-		rotate_ccw(tmp, mask, vc->vc_font.width, vc->vc_font.height);
+		font_glyph_rotate_270(tmp, vc->vc_font.width, vc->vc_font.height, mask);
 		kfree(tmp);
 	}
 
diff --git a/drivers/video/fbdev/core/fbcon_cw.c b/drivers/video/fbdev/core/fbcon_cw.c
index ea712654edae..5690fc1d7854 100644
--- a/drivers/video/fbdev/core/fbcon_cw.c
+++ b/drivers/video/fbdev/core/fbcon_cw.c
@@ -12,6 +12,7 @@
 #include <linux/slab.h>
 #include <linux/string.h>
 #include <linux/fb.h>
+#include <linux/font.h>
 #include <linux/vt_kern.h>
 #include <linux/console.h>
 #include <asm/types.h>
@@ -327,8 +328,7 @@ static void cw_cursor(struct vc_data *vc, struct fb_info *info, bool enable,
 		size = cur_height * width;
 		while (size--)
 			tmp[i++] = 0xff;
-		memset(mask, 0, w * vc->vc_font.width);
-		rotate_cw(tmp, mask, vc->vc_font.width, vc->vc_font.height);
+		font_glyph_rotate_90(tmp, vc->vc_font.width, vc->vc_font.height, mask);
 		kfree(tmp);
 	}
 
diff --git a/drivers/video/fbdev/core/fbcon_rotate.c b/drivers/video/fbdev/core/fbcon_rotate.c
index 18575c5182db..588dc9d6758a 100644
--- a/drivers/video/fbdev/core/fbcon_rotate.c
+++ b/drivers/video/fbdev/core/fbcon_rotate.c
@@ -12,6 +12,7 @@
 #include <linux/slab.h>
 #include <linux/string.h>
 #include <linux/fb.h>
+#include <linux/font.h>
 #include <linux/vt_kern.h>
 #include <linux/console.h>
 #include <asm/types.h>
@@ -60,30 +61,25 @@ int fbcon_rotate_font(struct fb_info *info, struct vc_data *vc)
 	}
 
 	dst = par->fontbuffer;
-	memset(dst, 0, par->fd_size);
 
 	switch (par->rotate) {
 	case FB_ROTATE_UD:
 		for (i = len; i--; ) {
-			rotate_ud(src, dst, vc->vc_font.width,
-				  vc->vc_font.height);
-
+			font_glyph_rotate_180(src, vc->vc_font.width, vc->vc_font.height, dst);
 			src += s_cellsize;
 			dst += d_cellsize;
 		}
 		break;
 	case FB_ROTATE_CW:
 		for (i = len; i--; ) {
-			rotate_cw(src, dst, vc->vc_font.width,
-				  vc->vc_font.height);
+			font_glyph_rotate_90(src, vc->vc_font.width, vc->vc_font.height, dst);
 			src += s_cellsize;
 			dst += d_cellsize;
 		}
 		break;
 	case FB_ROTATE_CCW:
 		for (i = len; i--; ) {
-			rotate_ccw(src, dst, vc->vc_font.width,
-				   vc->vc_font.height);
+			font_glyph_rotate_270(src, vc->vc_font.width, vc->vc_font.height, dst);
 			src += s_cellsize;
 			dst += d_cellsize;
 		}
diff --git a/drivers/video/fbdev/core/fbcon_rotate.h b/drivers/video/fbdev/core/fbcon_rotate.h
index 8cb019e8a9c0..725bcae2df61 100644
--- a/drivers/video/fbdev/core/fbcon_rotate.h
+++ b/drivers/video/fbdev/core/fbcon_rotate.h
@@ -19,77 +19,6 @@
         (fb_scrollmode(s) == SCROLL_REDRAW || fb_scrollmode(s) == SCROLL_MOVE || !(i)->fix.xpanstep) ? \
         (i)->var.xres : (i)->var.xres_virtual; })
 
-
-static inline int pattern_test_bit(u32 x, u32 y, u32 pitch, const char *pat)
-{
-	u32 tmp = (y * pitch) + x, index = tmp / 8,  bit = tmp % 8;
-
-	pat +=index;
-	return (*pat) & (0x80 >> bit);
-}
-
-static inline void pattern_set_bit(u32 x, u32 y, u32 pitch, char *pat)
-{
-	u32 tmp = (y * pitch) + x, index = tmp / 8, bit = tmp % 8;
-
-	pat += index;
-
-	(*pat) |= 0x80 >> bit;
-}
-
-static inline void rotate_ud(const char *in, char *out, u32 width, u32 height)
-{
-	int i, j;
-	int shift = (8 - (width % 8)) & 7;
-
-	width = (width + 7) & ~7;
-
-	for (i = 0; i < height; i++) {
-		for (j = 0; j < width - shift; j++) {
-			if (pattern_test_bit(j, i, width, in))
-				pattern_set_bit(width - (1 + j + shift),
-						height - (1 + i),
-						width, out);
-		}
-
-	}
-}
-
-static inline void rotate_cw(const char *in, char *out, u32 width, u32 height)
-{
-	int i, j, h = height, w = width;
-	int shift = (8 - (height % 8)) & 7;
-
-	width = (width + 7) & ~7;
-	height = (height + 7) & ~7;
-
-	for (i = 0; i < h; i++) {
-		for (j = 0; j < w; j++) {
-			if (pattern_test_bit(j, i, width, in))
-				pattern_set_bit(height - 1 - i - shift, j,
-						height, out);
-
-		}
-	}
-}
-
-static inline void rotate_ccw(const char *in, char *out, u32 width, u32 height)
-{
-	int i, j, h = height, w = width;
-	int shift = (8 - (width % 8)) & 7;
-
-	width = (width + 7) & ~7;
-	height = (height + 7) & ~7;
-
-	for (i = 0; i < h; i++) {
-		for (j = 0; j < w; j++) {
-			if (pattern_test_bit(j, i, width, in))
-				pattern_set_bit(i, width - 1 - j - shift,
-						height, out);
-		}
-	}
-}
-
 int fbcon_rotate_font(struct fb_info *info, struct vc_data *vc);
 
 #if defined(CONFIG_FRAMEBUFFER_CONSOLE_ROTATION)
diff --git a/include/linux/font.h b/include/linux/font.h
index 3bd49d914b22..0a240dd70422 100644
--- a/include/linux/font.h
+++ b/include/linux/font.h
@@ -104,6 +104,14 @@ unsigned int font_data_size(font_data_t *fd);
 bool font_data_is_equal(font_data_t *lhs, font_data_t *rhs);
 int font_data_export(font_data_t *fd, struct console_font *font, unsigned int vpitch);
 
+/* font_rotate.c */
+void font_glyph_rotate_90(const unsigned char *glyph, unsigned int width, unsigned int height,
+			  unsigned char *out);
+void font_glyph_rotate_180(const unsigned char *glyph, unsigned int width, unsigned int height,
+			   unsigned char *out);
+void font_glyph_rotate_270(const unsigned char *glyph, unsigned int width, unsigned int height,
+			   unsigned char *out);
+
 /*
  * Font description
  */
diff --git a/lib/fonts/Makefile b/lib/fonts/Makefile
index b176af53d53e..7202a70a56ef 100644
--- a/lib/fonts/Makefile
+++ b/lib/fonts/Makefile
@@ -2,6 +2,7 @@
 # Font handling
 
 font-y := fonts.o
+font-$(CONFIG_FRAMEBUFFER_CONSOLE_ROTATION) += font_rotate.o
 
 # Built-in fonts; sorted by Family-Size in ascending order
 font-$(CONFIG_FONT_6x8)       += font_6x8.o
diff --git a/lib/fonts/font_rotate.c b/lib/fonts/font_rotate.c
new file mode 100644
index 000000000000..d107a8d0a2b0
--- /dev/null
+++ b/lib/fonts/font_rotate.c
@@ -0,0 +1,150 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Font rotation
+ *
+ *    Copyright (C) 2005 Antonino Daplas <adaplas @pol.net>
+ *
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License.  See the file COPYING in the main directory of this archive for
+ * more details.
+ */
+
+#include <linux/export.h>
+#include <linux/math.h>
+#include <linux/string.h>
+
+#include "font.h"
+
+static inline int pattern_test_bit(u32 x, u32 y, u32 pitch, const char *pat)
+{
+	u32 tmp = (y * pitch) + x, index = tmp / 8,  bit = tmp % 8;
+
+	pat += index;
+	return (*pat) & (0x80 >> bit);
+}
+
+static inline void pattern_set_bit(u32 x, u32 y, u32 pitch, char *pat)
+{
+	u32 tmp = (y * pitch) + x, index = tmp / 8, bit = tmp % 8;
+
+	pat += index;
+
+	(*pat) |= 0x80 >> bit;
+}
+
+static inline void rotate_cw(const char *in, char *out, u32 width, u32 height)
+{
+	int i, j, h = height, w = width;
+	int shift = (8 - (height % 8)) & 7;
+
+	width = (width + 7) & ~7;
+	height = (height + 7) & ~7;
+
+	for (i = 0; i < h; i++) {
+		for (j = 0; j < w; j++) {
+			if (pattern_test_bit(j, i, width, in))
+				pattern_set_bit(height - 1 - i - shift, j,
+						height, out);
+		}
+	}
+}
+
+/**
+ * font_glyph_rotate_90 - Rotate a glyph pattern by 90° in clockwise direction
+ * @glyph: The glyph to rotate
+ * @width: The glyph width in bits per scanline
+ * @height: The number of scanlines in the glyph
+ * @out: The rotated glyph bitmap
+ *
+ * The parameters @width and @height refer to the input glyph given in @glyph.
+ * The caller has to provide the output buffer @out of sufficient size to hold
+ * the rotated glyph. Rotating by 90° flips the width and height for the output
+ * glyph. Depending on the glyph pitch, the size of the output glyph can be
+ * different than the size of the input. Callers have to take this into account
+ * when allocating the output memory.
+ */
+void font_glyph_rotate_90(const unsigned char *glyph, unsigned int width, unsigned int height,
+			  unsigned char *out)
+{
+	memset(out, 0, font_glyph_size(height, width)); /* flip width/height */
+
+	rotate_cw(glyph, out, width, height);
+}
+EXPORT_SYMBOL_GPL(font_glyph_rotate_90);
+
+static inline void rotate_ud(const char *in, char *out, u32 width, u32 height)
+{
+	int i, j;
+	int shift = (8 - (width % 8)) & 7;
+
+	width = (width + 7) & ~7;
+
+	for (i = 0; i < height; i++) {
+		for (j = 0; j < width - shift; j++) {
+			if (pattern_test_bit(j, i, width, in))
+				pattern_set_bit(width - (1 + j + shift),
+						height - (1 + i),
+						width, out);
+		}
+	}
+}
+
+/**
+ * font_glyph_rotate_180 - Rotate a glyph pattern by 180°
+ * @glyph: The glyph to rotate
+ * @width: The glyph width in bits per scanline
+ * @height: The number of scanlines in the glyph
+ * @out: The rotated glyph bitmap
+ *
+ * The parameters @width and @height refer to the input glyph given in @glyph.
+ * The caller has to provide the output buffer @out of sufficient size to hold
+ * the rotated glyph.
+ */
+void font_glyph_rotate_180(const unsigned char *glyph, unsigned int width, unsigned int height,
+			   unsigned char *out)
+{
+	memset(out, 0, font_glyph_size(width, height));
+
+	rotate_ud(glyph, out, width, height);
+}
+EXPORT_SYMBOL_GPL(font_glyph_rotate_180);
+
+static inline void rotate_ccw(const char *in, char *out, u32 width, u32 height)
+{
+	int i, j, h = height, w = width;
+	int shift = (8 - (width % 8)) & 7;
+
+	width = (width + 7) & ~7;
+	height = (height + 7) & ~7;
+
+	for (i = 0; i < h; i++) {
+		for (j = 0; j < w; j++) {
+			if (pattern_test_bit(j, i, width, in))
+				pattern_set_bit(i, width - 1 - j - shift,
+						height, out);
+		}
+	}
+}
+
+/**
+ * font_glyph_rotate_270 - Rotate a glyph pattern by 270° in clockwise direction
+ * @glyph: The glyph to rotate
+ * @width: The glyph width in bits per scanline
+ * @height: The number of scanlines in the glyph
+ * @out: The rotated glyph bitmap
+ *
+ * The parameters @width and @height refer to the input glyph given in @glyph.
+ * The caller has to provide the output buffer @out of sufficient size to hold
+ * the rotated glyph. Rotating by 270° flips the width and height for the output
+ * glyph. Depending on the glyph pitch, the size of the output glyph can be
+ * different than the size of the input. Callers have to take this into account
+ * when allocating the output memory.
+ */
+void font_glyph_rotate_270(const unsigned char *glyph, unsigned int width, unsigned int height,
+			   unsigned char *out)
+{
+	memset(out, 0, font_glyph_size(height, width)); /* flip width/height */
+
+	rotate_ccw(glyph, out, width, height);
+}
+EXPORT_SYMBOL_GPL(font_glyph_rotate_270);
-- 
2.53.0


^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH v2 06/10] lib/fonts: Refactor glyph-pattern helpers
  2026-04-07  9:23 [PATCH v2 00/10] fbcon,fonts: Refactor framebuffer console rotation Thomas Zimmermann
                   ` (4 preceding siblings ...)
  2026-04-07  9:23 ` [PATCH v2 05/10] lib/fonts: Implement glyph rotation Thomas Zimmermann
@ 2026-04-07  9:23 ` Thomas Zimmermann
  2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
  2026-04-07  9:23 ` [PATCH v2 07/10] lib/fonts: Refactor glyph-rotation helpers Thomas Zimmermann
                   ` (4 subsequent siblings)
  10 siblings, 1 reply; 23+ messages in thread
From: Thomas Zimmermann @ 2026-04-07  9:23 UTC (permalink / raw)
  To: deller, gregkh, jirislaby, geert, simona, sam
  Cc: linux-fbdev, dri-devel, linux-kernel, linux-serial,
	Thomas Zimmermann

Change the signatures of the pattern helpers to align them with other
font-glyph helpers: use the font_glyph_ prefix and pass the glyph
buffer first.

Calculating the position of the involved bit is somewhat obfuscated
in the original implementation. Move it into the new helper
__font_glyph_pos() and use the result as array index and bit position.

Note that these bit helpers use a bit pitch, while other code uses a
byte pitch. This is intentional and required here.

v2:
- fix typos in commit message

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 lib/fonts/font_rotate.c | 45 ++++++++++++++++++++++++-----------------
 1 file changed, 27 insertions(+), 18 deletions(-)

diff --git a/lib/fonts/font_rotate.c b/lib/fonts/font_rotate.c
index d107a8d0a2b0..d79ec5eef712 100644
--- a/lib/fonts/font_rotate.c
+++ b/lib/fonts/font_rotate.c
@@ -15,21 +15,33 @@
 
 #include "font.h"
 
-static inline int pattern_test_bit(u32 x, u32 y, u32 pitch, const char *pat)
+static unsigned int __font_glyph_pos(unsigned int x, unsigned int y, unsigned int bit_pitch,
+				     unsigned int *bit)
 {
-	u32 tmp = (y * pitch) + x, index = tmp / 8,  bit = tmp % 8;
+	unsigned int off = y * bit_pitch + x;
+	unsigned int bit_shift = off % 8;
 
-	pat += index;
-	return (*pat) & (0x80 >> bit);
+	*bit = 0x80 >> bit_shift; /* MSB has position 0, LSB has position 7 */
+
+	return off / 8;
 }
 
-static inline void pattern_set_bit(u32 x, u32 y, u32 pitch, char *pat)
+static bool font_glyph_test_bit(const unsigned char *glyph, unsigned int x, unsigned int y,
+				unsigned int bit_pitch)
 {
-	u32 tmp = (y * pitch) + x, index = tmp / 8, bit = tmp % 8;
+	unsigned int bit;
+	unsigned int i = __font_glyph_pos(x, y, bit_pitch, &bit);
+
+	return glyph[i] & bit;
+}
 
-	pat += index;
+static void font_glyph_set_bit(unsigned char *glyph, unsigned int x, unsigned int y,
+			       unsigned int bit_pitch)
+{
+	unsigned int bit;
+	unsigned int i = __font_glyph_pos(x, y, bit_pitch, &bit);
 
-	(*pat) |= 0x80 >> bit;
+	glyph[i] |= bit;
 }
 
 static inline void rotate_cw(const char *in, char *out, u32 width, u32 height)
@@ -42,9 +54,8 @@ static inline void rotate_cw(const char *in, char *out, u32 width, u32 height)
 
 	for (i = 0; i < h; i++) {
 		for (j = 0; j < w; j++) {
-			if (pattern_test_bit(j, i, width, in))
-				pattern_set_bit(height - 1 - i - shift, j,
-						height, out);
+			if (font_glyph_test_bit(in, j, i, width))
+				font_glyph_set_bit(out, height - 1 - i - shift, j, height);
 		}
 	}
 }
@@ -81,10 +92,9 @@ static inline void rotate_ud(const char *in, char *out, u32 width, u32 height)
 
 	for (i = 0; i < height; i++) {
 		for (j = 0; j < width - shift; j++) {
-			if (pattern_test_bit(j, i, width, in))
-				pattern_set_bit(width - (1 + j + shift),
-						height - (1 + i),
-						width, out);
+			if (font_glyph_test_bit(in, j, i, width))
+				font_glyph_set_bit(out, width - (1 + j + shift),
+						   height - (1 + i), width);
 		}
 	}
 }
@@ -119,9 +129,8 @@ static inline void rotate_ccw(const char *in, char *out, u32 width, u32 height)
 
 	for (i = 0; i < h; i++) {
 		for (j = 0; j < w; j++) {
-			if (pattern_test_bit(j, i, width, in))
-				pattern_set_bit(i, width - 1 - j - shift,
-						height, out);
+			if (font_glyph_test_bit(in, j, i, width))
+				font_glyph_set_bit(out, i, width - 1 - j - shift, height);
 		}
 	}
 }
-- 
2.53.0


^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH v2 07/10] lib/fonts: Refactor glyph-rotation helpers
  2026-04-07  9:23 [PATCH v2 00/10] fbcon,fonts: Refactor framebuffer console rotation Thomas Zimmermann
                   ` (5 preceding siblings ...)
  2026-04-07  9:23 ` [PATCH v2 06/10] lib/fonts: Refactor glyph-pattern helpers Thomas Zimmermann
@ 2026-04-07  9:23 ` Thomas Zimmermann
  2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
  2026-04-07  9:23 ` [PATCH v2 08/10] lib/fonts: Implement font rotation Thomas Zimmermann
                   ` (3 subsequent siblings)
  10 siblings, 1 reply; 23+ messages in thread
From: Thomas Zimmermann @ 2026-04-07  9:23 UTC (permalink / raw)
  To: deller, gregkh, jirislaby, geert, simona, sam
  Cc: linux-fbdev, dri-devel, linux-kernel, linux-serial,
	Thomas Zimmermann

Change the signatures of the glyph-rotation helpers to match their
public interfaces. Drop the inline qualifier.

Rename several variables to better match their meaning. Especially
rename variables to bit_pitch (or a variant thereof) if they store
a pitch value in bits per scanline. The original code is fairly
confusing about this. Move the calculation of the bit pitch into the
new helper font_glyph_bit_pitch().

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 lib/fonts/font_rotate.c | 85 ++++++++++++++++++++++++-----------------
 1 file changed, 49 insertions(+), 36 deletions(-)

diff --git a/lib/fonts/font_rotate.c b/lib/fonts/font_rotate.c
index d79ec5eef712..09f6218e036f 100644
--- a/lib/fonts/font_rotate.c
+++ b/lib/fonts/font_rotate.c
@@ -15,6 +15,12 @@
 
 #include "font.h"
 
+/* number of bits per line */
+static unsigned int font_glyph_bit_pitch(unsigned int width)
+{
+	return round_up(width, 8);
+}
+
 static unsigned int __font_glyph_pos(unsigned int x, unsigned int y, unsigned int bit_pitch,
 				     unsigned int *bit)
 {
@@ -44,18 +50,21 @@ static void font_glyph_set_bit(unsigned char *glyph, unsigned int x, unsigned in
 	glyph[i] |= bit;
 }
 
-static inline void rotate_cw(const char *in, char *out, u32 width, u32 height)
+static void __font_glyph_rotate_90(const unsigned char *glyph,
+				   unsigned int width, unsigned int height,
+				   unsigned char *out)
 {
-	int i, j, h = height, w = width;
-	int shift = (8 - (height % 8)) & 7;
-
-	width = (width + 7) & ~7;
-	height = (height + 7) & ~7;
-
-	for (i = 0; i < h; i++) {
-		for (j = 0; j < w; j++) {
-			if (font_glyph_test_bit(in, j, i, width))
-				font_glyph_set_bit(out, height - 1 - i - shift, j, height);
+	unsigned int x, y;
+	unsigned int shift = (8 - (height % 8)) & 7;
+	unsigned int bit_pitch = font_glyph_bit_pitch(width);
+	unsigned int out_bit_pitch = font_glyph_bit_pitch(height);
+
+	for (y = 0; y < height; y++) {
+		for (x = 0; x < width; x++) {
+			if (font_glyph_test_bit(glyph, x, y, bit_pitch)) {
+				font_glyph_set_bit(out, out_bit_pitch - 1 - y - shift, x,
+						   out_bit_pitch);
+			}
 		}
 	}
 }
@@ -79,22 +88,24 @@ void font_glyph_rotate_90(const unsigned char *glyph, unsigned int width, unsign
 {
 	memset(out, 0, font_glyph_size(height, width)); /* flip width/height */
 
-	rotate_cw(glyph, out, width, height);
+	__font_glyph_rotate_90(glyph, width, height, out);
 }
 EXPORT_SYMBOL_GPL(font_glyph_rotate_90);
 
-static inline void rotate_ud(const char *in, char *out, u32 width, u32 height)
+static void __font_glyph_rotate_180(const unsigned char *glyph,
+				    unsigned int width, unsigned int height,
+				    unsigned char *out)
 {
-	int i, j;
-	int shift = (8 - (width % 8)) & 7;
-
-	width = (width + 7) & ~7;
-
-	for (i = 0; i < height; i++) {
-		for (j = 0; j < width - shift; j++) {
-			if (font_glyph_test_bit(in, j, i, width))
-				font_glyph_set_bit(out, width - (1 + j + shift),
-						   height - (1 + i), width);
+	unsigned int x, y;
+	unsigned int shift = (8 - (width % 8)) & 7;
+	unsigned int bit_pitch = font_glyph_bit_pitch(width);
+
+	for (y = 0; y < height; y++) {
+		for (x = 0; x < width; x++) {
+			if (font_glyph_test_bit(glyph, x, y, bit_pitch)) {
+				font_glyph_set_bit(out, width - (1 + x + shift), height - (1 + y),
+						   bit_pitch);
+			}
 		}
 	}
 }
@@ -115,22 +126,24 @@ void font_glyph_rotate_180(const unsigned char *glyph, unsigned int width, unsig
 {
 	memset(out, 0, font_glyph_size(width, height));
 
-	rotate_ud(glyph, out, width, height);
+	__font_glyph_rotate_180(glyph, width, height, out);
 }
 EXPORT_SYMBOL_GPL(font_glyph_rotate_180);
 
-static inline void rotate_ccw(const char *in, char *out, u32 width, u32 height)
+static void __font_glyph_rotate_270(const unsigned char *glyph,
+				    unsigned int width, unsigned int height,
+				    unsigned char *out)
 {
-	int i, j, h = height, w = width;
-	int shift = (8 - (width % 8)) & 7;
-
-	width = (width + 7) & ~7;
-	height = (height + 7) & ~7;
-
-	for (i = 0; i < h; i++) {
-		for (j = 0; j < w; j++) {
-			if (font_glyph_test_bit(in, j, i, width))
-				font_glyph_set_bit(out, i, width - 1 - j - shift, height);
+	unsigned int x, y;
+	unsigned int shift = (8 - (width % 8)) & 7;
+	unsigned int bit_pitch = font_glyph_bit_pitch(width);
+	unsigned int out_bit_pitch = font_glyph_bit_pitch(height);
+
+	for (y = 0; y < height; y++) {
+		for (x = 0; x < width; x++) {
+			if (font_glyph_test_bit(glyph, x, y, bit_pitch))
+				font_glyph_set_bit(out, y, bit_pitch - 1 - x - shift,
+						   out_bit_pitch);
 		}
 	}
 }
@@ -154,6 +167,6 @@ void font_glyph_rotate_270(const unsigned char *glyph, unsigned int width, unsig
 {
 	memset(out, 0, font_glyph_size(height, width)); /* flip width/height */
 
-	rotate_ccw(glyph, out, width, height);
+	__font_glyph_rotate_270(glyph, width, height, out);
 }
 EXPORT_SYMBOL_GPL(font_glyph_rotate_270);
-- 
2.53.0


^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH v2 08/10] lib/fonts: Implement font rotation
  2026-04-07  9:23 [PATCH v2 00/10] fbcon,fonts: Refactor framebuffer console rotation Thomas Zimmermann
                   ` (6 preceding siblings ...)
  2026-04-07  9:23 ` [PATCH v2 07/10] lib/fonts: Refactor glyph-rotation helpers Thomas Zimmermann
@ 2026-04-07  9:23 ` Thomas Zimmermann
  2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
  2026-04-07  9:23 ` [PATCH v2 09/10] fbcon: Fill cursor mask in helper function Thomas Zimmermann
                   ` (2 subsequent siblings)
  10 siblings, 1 reply; 23+ messages in thread
From: Thomas Zimmermann @ 2026-04-07  9:23 UTC (permalink / raw)
  To: deller, gregkh, jirislaby, geert, simona, sam
  Cc: linux-fbdev, dri-devel, linux-kernel, linux-serial,
	Thomas Zimmermann

Move the core of fbcon's font-rotation code to the font library as
the new helper font_data_rotate(). The code can rotate in steps of
90°. For completeness, it also copies the glyph data for multiples
of 360°.

Bring back the memset optimization. A memset to 0 again clears the
whole glyph output buffer. Then use the internal rotation helpers on
the cleared output. Fbcon's original implementation worked like this,
but lost it during refactoring.

Replace fbcon's font-rotation code with the new implementations.
All that's left to do for fbcon is to maintain its internal fbcon
state.

v2:
- fix typos

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/video/fbdev/core/fbcon.h        |   2 +-
 drivers/video/fbdev/core/fbcon_rotate.c |  78 +++++-------------
 include/linux/font.h                    |   3 +
 lib/fonts/font_rotate.c                 | 103 ++++++++++++++++++++++++
 4 files changed, 126 insertions(+), 60 deletions(-)

diff --git a/drivers/video/fbdev/core/fbcon.h b/drivers/video/fbdev/core/fbcon.h
index 1e3c1ef84762..1793f34a6c84 100644
--- a/drivers/video/fbdev/core/fbcon.h
+++ b/drivers/video/fbdev/core/fbcon.h
@@ -86,7 +86,7 @@ struct fbcon_par {
 	const u8    *fontdata;
 	u8    *cursor_src;
 	u32    cursor_size;
-	u32    fd_size;
+	size_t fd_size;
 
 	const struct fbcon_bitops *bitops;
 };
diff --git a/drivers/video/fbdev/core/fbcon_rotate.c b/drivers/video/fbdev/core/fbcon_rotate.c
index 588dc9d6758a..74206f5a6e98 100644
--- a/drivers/video/fbdev/core/fbcon_rotate.c
+++ b/drivers/video/fbdev/core/fbcon_rotate.c
@@ -8,84 +8,44 @@
  *  more details.
  */
 
-#include <linux/module.h>
-#include <linux/slab.h>
-#include <linux/string.h>
+#include <linux/errno.h>
 #include <linux/fb.h>
 #include <linux/font.h>
-#include <linux/vt_kern.h>
-#include <linux/console.h>
-#include <asm/types.h>
+
 #include "fbcon.h"
 #include "fbcon_rotate.h"
 
 int fbcon_rotate_font(struct fb_info *info, struct vc_data *vc)
 {
 	struct fbcon_par *par = info->fbcon_par;
-	int len, err = 0;
-	int s_cellsize, d_cellsize, i;
-	const u8 *src;
-	u8 *dst;
+	unsigned char *fontbuffer;
+	int ret;
 
 	if (vc->vc_font.data == par->fontdata &&
 	    par->p->con_rotate == par->cur_rotate)
-		goto finished;
+		return 0;
 
-	src = par->fontdata = vc->vc_font.data;
+	par->fontdata = vc->vc_font.data;
 	par->cur_rotate = par->p->con_rotate;
-	len = vc->vc_font.charcount;
-	s_cellsize = font_glyph_size(vc->vc_font.width, vc->vc_font.height);
-	d_cellsize = s_cellsize;
-
-	if (par->rotate == FB_ROTATE_CW ||
-	    par->rotate == FB_ROTATE_CCW)
-		d_cellsize = font_glyph_size(vc->vc_font.height, vc->vc_font.width);
 
 	if (info->fbops->fb_sync)
 		info->fbops->fb_sync(info);
 
-	if (par->fd_size < d_cellsize * len) {
-		kfree(par->fontbuffer);
-		par->fontbuffer = NULL;
-		par->fd_size = 0;
-
-		dst = kmalloc_array(len, d_cellsize, GFP_KERNEL);
-
-		if (dst == NULL) {
-			err = -ENOMEM;
-			goto finished;
-		}
-
-		par->fd_size = d_cellsize * len;
-		par->fontbuffer = dst;
+	fontbuffer = font_data_rotate(par->p->fontdata, vc->vc_font.width,
+				      vc->vc_font.height, vc->vc_font.charcount,
+				      par->rotate, par->fontbuffer, &par->fd_size);
+	if (IS_ERR(fontbuffer)) {
+		ret = PTR_ERR(fontbuffer);
+		goto err_kfree;
 	}
 
-	dst = par->fontbuffer;
+	par->fontbuffer = fontbuffer;
 
-	switch (par->rotate) {
-	case FB_ROTATE_UD:
-		for (i = len; i--; ) {
-			font_glyph_rotate_180(src, vc->vc_font.width, vc->vc_font.height, dst);
-			src += s_cellsize;
-			dst += d_cellsize;
-		}
-		break;
-	case FB_ROTATE_CW:
-		for (i = len; i--; ) {
-			font_glyph_rotate_90(src, vc->vc_font.width, vc->vc_font.height, dst);
-			src += s_cellsize;
-			dst += d_cellsize;
-		}
-		break;
-	case FB_ROTATE_CCW:
-		for (i = len; i--; ) {
-			font_glyph_rotate_270(src, vc->vc_font.width, vc->vc_font.height, dst);
-			src += s_cellsize;
-			dst += d_cellsize;
-		}
-		break;
-	}
+	return 0;
+
+err_kfree:
+	kfree(par->fontbuffer);
+	par->fontbuffer = NULL; /* clear here to avoid output */
 
-finished:
-	return err;
+	return ret;
 }
diff --git a/include/linux/font.h b/include/linux/font.h
index 0a240dd70422..6845f02d739a 100644
--- a/include/linux/font.h
+++ b/include/linux/font.h
@@ -111,6 +111,9 @@ void font_glyph_rotate_180(const unsigned char *glyph, unsigned int width, unsig
 			   unsigned char *out);
 void font_glyph_rotate_270(const unsigned char *glyph, unsigned int width, unsigned int height,
 			   unsigned char *out);
+unsigned char *font_data_rotate(font_data_t *fd, unsigned int width, unsigned int height,
+				unsigned int charcount, unsigned int steps,
+				unsigned char *buf, size_t *bufsize);
 
 /*
  * Font description
diff --git a/lib/fonts/font_rotate.c b/lib/fonts/font_rotate.c
index 09f6218e036f..065e0fc0667b 100644
--- a/lib/fonts/font_rotate.c
+++ b/lib/fonts/font_rotate.c
@@ -9,8 +9,11 @@
  * more details.
  */
 
+#include <linux/errno.h>
 #include <linux/export.h>
 #include <linux/math.h>
+#include <linux/overflow.h>
+#include <linux/slab.h>
 #include <linux/string.h>
 
 #include "font.h"
@@ -170,3 +173,103 @@ void font_glyph_rotate_270(const unsigned char *glyph, unsigned int width, unsig
 	__font_glyph_rotate_270(glyph, width, height, out);
 }
 EXPORT_SYMBOL_GPL(font_glyph_rotate_270);
+
+/**
+ * font_data_rotate - Rotate font data by multiples of 90°
+ * @fd: The font data to rotate
+ * @width: The glyph width in bits per scanline
+ * @height: The number of scanlines in the glyph
+ * @charcount: The number of glyphs in the font
+ * @steps: Number of rotation steps of 90°
+ * @buf: Preallocated output buffer; can be NULL
+ * @bufsize: The size of @buf in bytes; can be NULL
+ *
+ * The parameters @width and @height refer to the visible number of pixels
+ * and scanlines in a single glyph. The number of glyphs is given in @charcount.
+ * Rotation happens in steps of 90°. The @steps parameter can have any value,
+ * but only 0 to 3 produce distinct results. With 4 or higher, a full rotation
+ * has been performed. You can pass any value for @steps and the helper will
+ * perform the appropriate rotation. Note that the returned buffer is not
+ * compatible with font_data_t. It only contains glyph data in the same format
+ * as returned by font_data_buf(). Callers are responsible to free the returned
+ * buffer with kfree(). Font rotation typically happens when displays get
+ * re-oriented. To avoid unnecessary re-allocation of the memory buffer, the
+ * caller can pass in an earlier result buffer in @buf for reuse. The old and
+ * new buffer sizes are given and retrieved by the caller in @bufsize. The
+ * allocation semantics are compatible with krealloc().
+ *
+ * Returns:
+ * A buffer with rotated glyphs on success, or an error pointer otherwise
+ */
+unsigned char *font_data_rotate(font_data_t *fd, unsigned int width, unsigned int height,
+				unsigned int charcount, unsigned int steps,
+				unsigned char *buf, size_t *bufsize)
+{
+	const unsigned char *src = font_data_buf(fd);
+	unsigned int s_cellsize = font_glyph_size(width, height);
+	unsigned int d_cellsize, i;
+	unsigned char *dst;
+	size_t size;
+
+	steps %= 4;
+
+	switch (steps) {
+	case 0:
+	case 2:
+		d_cellsize = s_cellsize;
+		break;
+	case 1:
+	case 3:
+		d_cellsize = font_glyph_size(height, width); /* flip width/height */
+		break;
+	}
+
+	if (check_mul_overflow(charcount, d_cellsize, &size))
+		return ERR_PTR(-EINVAL);
+
+	if (!buf || !bufsize || size > *bufsize) {
+		dst = kmalloc_array(charcount, d_cellsize, GFP_KERNEL);
+		if (!dst)
+			return ERR_PTR(-ENOMEM);
+
+		kfree(buf);
+		buf = dst;
+		if (bufsize)
+			*bufsize = size;
+	} else {
+		dst = buf;
+	}
+
+	switch (steps) {
+	case 0:
+		memcpy(dst, src, size);
+		break;
+	case 1:
+		memset(dst, 0, size);
+		for (i = 0; i < charcount; ++i) {
+			__font_glyph_rotate_90(src, width, height, dst);
+			src += s_cellsize;
+			dst += d_cellsize;
+		}
+		break;
+	case 2:
+		memset(dst, 0, size);
+		for (i = 0; i < charcount; ++i) {
+			__font_glyph_rotate_180(src, width, height, dst);
+			src += s_cellsize;
+			dst += d_cellsize;
+		}
+		break;
+	case 3:
+		memset(dst, 0, size);
+		for (i = 0; i < charcount; ++i) {
+			__font_glyph_rotate_270(src, width, height, dst);
+			src += s_cellsize;
+			dst += d_cellsize;
+		}
+		break;
+	}
+
+	return buf;
+}
+EXPORT_SYMBOL_GPL(font_data_rotate);
-- 
2.53.0


^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH v2 09/10] fbcon: Fill cursor mask in helper function
  2026-04-07  9:23 [PATCH v2 00/10] fbcon,fonts: Refactor framebuffer console rotation Thomas Zimmermann
                   ` (7 preceding siblings ...)
  2026-04-07  9:23 ` [PATCH v2 08/10] lib/fonts: Implement font rotation Thomas Zimmermann
@ 2026-04-07  9:23 ` Thomas Zimmermann
  2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
  2026-04-07  9:23 ` [PATCH v2 10/10] fbcon: Put font-rotation state into separate struct Thomas Zimmermann
  2026-04-12  3:55 ` Claude review: fbcon,fonts: Refactor framebuffer console rotation Claude Code Review Bot
  10 siblings, 1 reply; 23+ messages in thread
From: Thomas Zimmermann @ 2026-04-07  9:23 UTC (permalink / raw)
  To: deller, gregkh, jirislaby, geert, simona, sam
  Cc: linux-fbdev, dri-devel, linux-kernel, linux-serial,
	Thomas Zimmermann

Fbcon creates a cursor shape on the fly from the user-configured
settings. The logic to create a glyph with the cursor's bitmap mask
is duplicated in four places. In the cases that involve console
rotation, the implementation further rotates the cursor glyph for
displaying.

Consolidate all cursor-mask creation in a single helper. Update the
callers accordingly. For console rotation, use the glyph helpers to
rotate the created cursor glyph to the correct orientation.

v2:
- fix sparse truncated-bits warning

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/video/fbdev/core/bitblit.c   | 35 ++-----------------
 drivers/video/fbdev/core/fbcon.c     | 40 ++++++++++++++++++++++
 drivers/video/fbdev/core/fbcon.h     |  2 ++
 drivers/video/fbdev/core/fbcon_ccw.c | 51 ++++++----------------------
 drivers/video/fbdev/core/fbcon_cw.c  | 51 ++++++----------------------
 drivers/video/fbdev/core/fbcon_ud.c  | 50 +++++++--------------------
 6 files changed, 78 insertions(+), 151 deletions(-)

diff --git a/drivers/video/fbdev/core/bitblit.c b/drivers/video/fbdev/core/bitblit.c
index 7478accea8ec..65681dcc5930 100644
--- a/drivers/video/fbdev/core/bitblit.c
+++ b/drivers/video/fbdev/core/bitblit.c
@@ -329,46 +329,17 @@ static void bit_cursor(struct vc_data *vc, struct fb_info *info, bool enable,
 	    vc->vc_cursor_type != par->p->cursor_shape ||
 	    par->cursor_state.mask == NULL ||
 	    par->cursor_reset) {
-		char *mask = kmalloc_array(w, vc->vc_font.height, GFP_ATOMIC);
-		int cur_height, size, i = 0;
-		u8 msk = 0xff;
+		unsigned char *mask = kmalloc_array(vc->vc_font.height, w, GFP_ATOMIC);
 
 		if (!mask)
 			return;
+		fbcon_fill_cursor_mask(par, vc, mask);
 
 		kfree(par->cursor_state.mask);
-		par->cursor_state.mask = mask;
+		par->cursor_state.mask = (const char *)mask;
 
 		par->p->cursor_shape = vc->vc_cursor_type;
 		cursor.set |= FB_CUR_SETSHAPE;
-
-		switch (CUR_SIZE(par->p->cursor_shape)) {
-		case CUR_NONE:
-			cur_height = 0;
-			break;
-		case CUR_UNDERLINE:
-			cur_height = (vc->vc_font.height < 10) ? 1 : 2;
-			break;
-		case CUR_LOWER_THIRD:
-			cur_height = vc->vc_font.height/3;
-			break;
-		case CUR_LOWER_HALF:
-			cur_height = vc->vc_font.height >> 1;
-			break;
-		case CUR_TWO_THIRDS:
-			cur_height = (vc->vc_font.height << 1)/3;
-			break;
-		case CUR_BLOCK:
-		default:
-			cur_height = vc->vc_font.height;
-			break;
-		}
-		size = (vc->vc_font.height - cur_height) * w;
-		while (size--)
-			mask[i++] = ~msk;
-		size = cur_height * w;
-		while (size--)
-			mask[i++] = msk;
 	}
 
 	par->cursor_state.enable = enable && !use_sw;
diff --git a/drivers/video/fbdev/core/fbcon.c b/drivers/video/fbdev/core/fbcon.c
index 8641b0b3edc4..ff4c69e971f8 100644
--- a/drivers/video/fbdev/core/fbcon.c
+++ b/drivers/video/fbdev/core/fbcon.c
@@ -446,6 +446,46 @@ static void fbcon_del_cursor_work(struct fb_info *info)
 	cancel_delayed_work_sync(&par->cursor_work);
 }
 
+void fbcon_fill_cursor_mask(struct fbcon_par *par, struct vc_data *vc, unsigned char *mask)
+{
+	static const unsigned int pattern = 0xffffffff;
+	unsigned int pitch = vc_font_pitch(&vc->vc_font);
+	unsigned int cur_height, size;
+
+	switch (CUR_SIZE(vc->vc_cursor_type)) {
+	case CUR_NONE:
+		cur_height = 0;
+		break;
+	case CUR_UNDERLINE:
+		if (vc->vc_font.height < 10)
+			cur_height = 1;
+		else
+			cur_height = 2;
+		break;
+	case CUR_LOWER_THIRD:
+		cur_height = vc->vc_font.height / 3;
+		break;
+	case CUR_LOWER_HALF:
+		cur_height = vc->vc_font.height / 2;
+		break;
+	case CUR_TWO_THIRDS:
+		cur_height = (vc->vc_font.height * 2) / 3;
+		break;
+	case CUR_BLOCK:
+	default:
+		cur_height = vc->vc_font.height;
+		break;
+	}
+
+	size = (vc->vc_font.height - cur_height) * pitch;
+	while (size--)
+		*mask++ = (unsigned char)~pattern;
+
+	size = cur_height * pitch;
+	while (size--)
+		*mask++ = (unsigned char)pattern;
+}
+
 #ifndef MODULE
 static int __init fb_console_setup(char *this_opt)
 {
diff --git a/drivers/video/fbdev/core/fbcon.h b/drivers/video/fbdev/core/fbcon.h
index 1793f34a6c84..bb0727b70631 100644
--- a/drivers/video/fbdev/core/fbcon.h
+++ b/drivers/video/fbdev/core/fbcon.h
@@ -192,6 +192,8 @@ extern void fbcon_set_tileops(struct vc_data *vc, struct fb_info *info);
 extern void fbcon_set_bitops_ur(struct fbcon_par *par);
 extern int  soft_cursor(struct fb_info *info, struct fb_cursor *cursor);
 
+void fbcon_fill_cursor_mask(struct fbcon_par *par, struct vc_data *vc, unsigned char *mask);
+
 #define FBCON_ATTRIBUTE_UNDERLINE 1
 #define FBCON_ATTRIBUTE_REVERSE   2
 #define FBCON_ATTRIBUTE_BOLD      4
diff --git a/drivers/video/fbdev/core/fbcon_ccw.c b/drivers/video/fbdev/core/fbcon_ccw.c
index 72453a2aaca8..723d9a33067f 100644
--- a/drivers/video/fbdev/core/fbcon_ccw.c
+++ b/drivers/video/fbdev/core/fbcon_ccw.c
@@ -296,57 +296,26 @@ static void ccw_cursor(struct vc_data *vc, struct fb_info *info, bool enable,
 	    vc->vc_cursor_type != par->p->cursor_shape ||
 	    par->cursor_state.mask == NULL ||
 	    par->cursor_reset) {
-		char *tmp, *mask = kmalloc_array(w, vc->vc_font.width,
-						 GFP_ATOMIC);
-		int cur_height, size, i = 0;
-		int width = font_glyph_pitch(vc->vc_font.width);
+		unsigned char *tmp, *mask;
 
-		if (!mask)
+		tmp = kmalloc_array(vc->vc_font.height, vc_font_pitch(&vc->vc_font), GFP_ATOMIC);
+		if (!tmp)
 			return;
+		fbcon_fill_cursor_mask(par, vc, tmp);
 
-		tmp = kmalloc_array(width, vc->vc_font.height, GFP_ATOMIC);
-
-		if (!tmp) {
-			kfree(mask);
+		mask = kmalloc_array(vc->vc_font.width, w, GFP_ATOMIC);
+		if (!mask) {
+			kfree(tmp);
 			return;
 		}
+		font_glyph_rotate_270(tmp, vc->vc_font.width, vc->vc_font.height, mask);
+		kfree(tmp);
 
 		kfree(par->cursor_state.mask);
-		par->cursor_state.mask = mask;
+		par->cursor_state.mask = (const char *)mask;
 
 		par->p->cursor_shape = vc->vc_cursor_type;
 		cursor.set |= FB_CUR_SETSHAPE;
-
-		switch (CUR_SIZE(par->p->cursor_shape)) {
-		case CUR_NONE:
-			cur_height = 0;
-			break;
-		case CUR_UNDERLINE:
-			cur_height = (vc->vc_font.height < 10) ? 1 : 2;
-			break;
-		case CUR_LOWER_THIRD:
-			cur_height = vc->vc_font.height/3;
-			break;
-		case CUR_LOWER_HALF:
-			cur_height = vc->vc_font.height >> 1;
-			break;
-		case CUR_TWO_THIRDS:
-			cur_height = (vc->vc_font.height << 1)/3;
-			break;
-		case CUR_BLOCK:
-		default:
-			cur_height = vc->vc_font.height;
-			break;
-		}
-
-		size = (vc->vc_font.height - cur_height) * width;
-		while (size--)
-			tmp[i++] = 0;
-		size = cur_height * width;
-		while (size--)
-			tmp[i++] = 0xff;
-		font_glyph_rotate_270(tmp, vc->vc_font.width, vc->vc_font.height, mask);
-		kfree(tmp);
 	}
 
 	par->cursor_state.enable = enable && !use_sw;
diff --git a/drivers/video/fbdev/core/fbcon_cw.c b/drivers/video/fbdev/core/fbcon_cw.c
index 5690fc1d7854..732d093d462f 100644
--- a/drivers/video/fbdev/core/fbcon_cw.c
+++ b/drivers/video/fbdev/core/fbcon_cw.c
@@ -279,57 +279,26 @@ static void cw_cursor(struct vc_data *vc, struct fb_info *info, bool enable,
 	    vc->vc_cursor_type != par->p->cursor_shape ||
 	    par->cursor_state.mask == NULL ||
 	    par->cursor_reset) {
-		char *tmp, *mask = kmalloc_array(w, vc->vc_font.width,
-						 GFP_ATOMIC);
-		int cur_height, size, i = 0;
-		int width = font_glyph_pitch(vc->vc_font.width);
+		unsigned char *tmp, *mask;
 
-		if (!mask)
+		tmp = kmalloc_array(vc->vc_font.height, vc_font_pitch(&vc->vc_font), GFP_ATOMIC);
+		if (!tmp)
 			return;
+		fbcon_fill_cursor_mask(par, vc, tmp);
 
-		tmp = kmalloc_array(width, vc->vc_font.height, GFP_ATOMIC);
-
-		if (!tmp) {
-			kfree(mask);
+		mask = kmalloc_array(vc->vc_font.width, w, GFP_ATOMIC);
+		if (!mask) {
+			kfree(tmp);
 			return;
 		}
+		font_glyph_rotate_90(tmp, vc->vc_font.width, vc->vc_font.height, mask);
+		kfree(tmp);
 
 		kfree(par->cursor_state.mask);
-		par->cursor_state.mask = mask;
+		par->cursor_state.mask = (const char *)mask;
 
 		par->p->cursor_shape = vc->vc_cursor_type;
 		cursor.set |= FB_CUR_SETSHAPE;
-
-		switch (CUR_SIZE(par->p->cursor_shape)) {
-		case CUR_NONE:
-			cur_height = 0;
-			break;
-		case CUR_UNDERLINE:
-			cur_height = (vc->vc_font.height < 10) ? 1 : 2;
-			break;
-		case CUR_LOWER_THIRD:
-			cur_height = vc->vc_font.height/3;
-			break;
-		case CUR_LOWER_HALF:
-			cur_height = vc->vc_font.height >> 1;
-			break;
-		case CUR_TWO_THIRDS:
-			cur_height = (vc->vc_font.height << 1)/3;
-			break;
-		case CUR_BLOCK:
-		default:
-			cur_height = vc->vc_font.height;
-			break;
-		}
-
-		size = (vc->vc_font.height - cur_height) * width;
-		while (size--)
-			tmp[i++] = 0;
-		size = cur_height * width;
-		while (size--)
-			tmp[i++] = 0xff;
-		font_glyph_rotate_90(tmp, vc->vc_font.width, vc->vc_font.height, mask);
-		kfree(tmp);
 	}
 
 	par->cursor_state.enable = enable && !use_sw;
diff --git a/drivers/video/fbdev/core/fbcon_ud.c b/drivers/video/fbdev/core/fbcon_ud.c
index f7cd89c42b01..a1981fa4701a 100644
--- a/drivers/video/fbdev/core/fbcon_ud.c
+++ b/drivers/video/fbdev/core/fbcon_ud.c
@@ -326,50 +326,26 @@ static void ud_cursor(struct vc_data *vc, struct fb_info *info, bool enable,
 	    vc->vc_cursor_type != par->p->cursor_shape ||
 	    par->cursor_state.mask == NULL ||
 	    par->cursor_reset) {
-		char *mask = kmalloc_array(w, vc->vc_font.height, GFP_ATOMIC);
-		int cur_height, size, i = 0;
-		u8 msk = 0xff;
+		unsigned char *tmp, *mask;
 
-		if (!mask)
+		tmp = kmalloc_array(vc->vc_font.height, w, GFP_ATOMIC);
+		if (!tmp)
 			return;
+		fbcon_fill_cursor_mask(par, vc, tmp);
+
+		mask = kmalloc_array(vc->vc_font.height, w, GFP_ATOMIC);
+		if (!mask) {
+			kfree(tmp);
+			return;
+		}
+		font_glyph_rotate_180(tmp, vc->vc_font.width, vc->vc_font.height, mask);
+		kfree(tmp);
 
 		kfree(par->cursor_state.mask);
-		par->cursor_state.mask = mask;
+		par->cursor_state.mask = (const char *)mask;
 
 		par->p->cursor_shape = vc->vc_cursor_type;
 		cursor.set |= FB_CUR_SETSHAPE;
-
-		switch (CUR_SIZE(par->p->cursor_shape)) {
-		case CUR_NONE:
-			cur_height = 0;
-			break;
-		case CUR_UNDERLINE:
-			cur_height = (vc->vc_font.height < 10) ? 1 : 2;
-			break;
-		case CUR_LOWER_THIRD:
-			cur_height = vc->vc_font.height/3;
-			break;
-		case CUR_LOWER_HALF:
-			cur_height = vc->vc_font.height >> 1;
-			break;
-		case CUR_TWO_THIRDS:
-			cur_height = (vc->vc_font.height << 1)/3;
-			break;
-		case CUR_BLOCK:
-		default:
-			cur_height = vc->vc_font.height;
-			break;
-		}
-
-		size = cur_height * w;
-
-		while (size--)
-			mask[i++] = msk;
-
-		size = (vc->vc_font.height - cur_height) * w;
-
-		while (size--)
-			mask[i++] = ~msk;
 	}
 
 	par->cursor_state.enable = enable && !use_sw;
-- 
2.53.0


^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH v2 10/10] fbcon: Put font-rotation state into separate struct
  2026-04-07  9:23 [PATCH v2 00/10] fbcon,fonts: Refactor framebuffer console rotation Thomas Zimmermann
                   ` (8 preceding siblings ...)
  2026-04-07  9:23 ` [PATCH v2 09/10] fbcon: Fill cursor mask in helper function Thomas Zimmermann
@ 2026-04-07  9:23 ` Thomas Zimmermann
  2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
  2026-04-12  3:55 ` Claude review: fbcon,fonts: Refactor framebuffer console rotation Claude Code Review Bot
  10 siblings, 1 reply; 23+ messages in thread
From: Thomas Zimmermann @ 2026-04-07  9:23 UTC (permalink / raw)
  To: deller, gregkh, jirislaby, geert, simona, sam
  Cc: linux-fbdev, dri-devel, linux-kernel, linux-serial,
	Thomas Zimmermann

Move all temporary state of the font-rotation code into the struct
rotated in struct fbcon_par. Protect it with the Kconfig symbol
CONFIG_FRAMEBUFFER_CONSOLE_ROTATION. Avoids mixing it up with fbcon's
regular state.

v2:
- fix typos

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
---
 drivers/video/fbdev/core/fbcon.c        |  8 ++++++--
 drivers/video/fbdev/core/fbcon.h        | 12 +++++++----
 drivers/video/fbdev/core/fbcon_ccw.c    |  8 ++++----
 drivers/video/fbdev/core/fbcon_cw.c     |  8 ++++----
 drivers/video/fbdev/core/fbcon_rotate.c | 27 +++++++++++++------------
 drivers/video/fbdev/core/fbcon_ud.c     | 10 ++++-----
 6 files changed, 41 insertions(+), 32 deletions(-)

diff --git a/drivers/video/fbdev/core/fbcon.c b/drivers/video/fbdev/core/fbcon.c
index ff4c69e971f8..b0e3e765360d 100644
--- a/drivers/video/fbdev/core/fbcon.c
+++ b/drivers/video/fbdev/core/fbcon.c
@@ -787,7 +787,9 @@ static void fbcon_release(struct fb_info *info)
 		kfree(par->cursor_state.mask);
 		kfree(par->cursor_data);
 		kfree(par->cursor_src);
-		kfree(par->fontbuffer);
+#ifdef CONFIG_FRAMEBUFFER_CONSOLE_ROTATION
+		kfree(par->rotated.buf);
+#endif
 		kfree(info->fbcon_par);
 		info->fbcon_par = NULL;
 	}
@@ -1040,7 +1042,9 @@ static const char *fbcon_startup(void)
 	par = info->fbcon_par;
 	par->currcon = -1;
 	par->graphics = 1;
-	par->cur_rotate = -1;
+#ifdef CONFIG_FRAMEBUFFER_CONSOLE_ROTATION
+	par->rotated.buf_rotate = -1;
+#endif
 
 	p->con_rotate = initial_rotation;
 	if (p->con_rotate == -1)
diff --git a/drivers/video/fbdev/core/fbcon.h b/drivers/video/fbdev/core/fbcon.h
index bb0727b70631..321cc7f44baa 100644
--- a/drivers/video/fbdev/core/fbcon.h
+++ b/drivers/video/fbdev/core/fbcon.h
@@ -80,13 +80,17 @@ struct fbcon_par {
 	int    graphics;
 	bool   initialized;
 	int    rotate;
-	int    cur_rotate;
 	char  *cursor_data;
-	u8          *fontbuffer;
-	const u8    *fontdata;
+#ifdef CONFIG_FRAMEBUFFER_CONSOLE_ROTATION
+	struct {
+		font_data_t *fontdata;  /* source font */
+		u8 *buf;                /* rotated glyphs */
+		size_t bufsize;
+		int buf_rotate;         /* rotation of buf */
+	} rotated;
+#endif
 	u8    *cursor_src;
 	u32    cursor_size;
-	size_t fd_size;
 
 	const struct fbcon_bitops *bitops;
 };
diff --git a/drivers/video/fbdev/core/fbcon_ccw.c b/drivers/video/fbdev/core/fbcon_ccw.c
index 723d9a33067f..33f02d579e02 100644
--- a/drivers/video/fbdev/core/fbcon_ccw.c
+++ b/drivers/video/fbdev/core/fbcon_ccw.c
@@ -106,7 +106,7 @@ static inline void ccw_putcs_aligned(struct vc_data *vc, struct fb_info *info,
 	u8 *src;
 
 	while (cnt--) {
-		src = par->fontbuffer + (scr_readw(s--) & charmask) * cellsize;
+		src = par->rotated.buf + (scr_readw(s--) & charmask) * cellsize;
 
 		if (attr) {
 			ccw_update_attr(buf, src, attr, vc);
@@ -142,7 +142,7 @@ static void ccw_putcs(struct vc_data *vc, struct fb_info *info,
 	u8 *dst, *buf = NULL;
 	u32 vyres = GETVYRES(par->p, info);
 
-	if (!par->fontbuffer)
+	if (!par->rotated.buf)
 		return;
 
 	image.fg_color = fg;
@@ -232,14 +232,14 @@ static void ccw_cursor(struct vc_data *vc, struct fb_info *info, bool enable,
 	char *src;
 	u32 vyres = GETVYRES(par->p, info);
 
-	if (!par->fontbuffer)
+	if (!par->rotated.buf)
 		return;
 
 	cursor.set = 0;
 
  	c = scr_readw((u16 *) vc->vc_pos);
 	attribute = get_attribute(info, c);
-	src = par->fontbuffer + ((c & charmask) * (w * vc->vc_font.width));
+	src = par->rotated.buf + ((c & charmask) * (w * vc->vc_font.width));
 
 	if (par->cursor_state.image.data != src ||
 	    par->cursor_reset) {
diff --git a/drivers/video/fbdev/core/fbcon_cw.c b/drivers/video/fbdev/core/fbcon_cw.c
index 732d093d462f..bde820967eb9 100644
--- a/drivers/video/fbdev/core/fbcon_cw.c
+++ b/drivers/video/fbdev/core/fbcon_cw.c
@@ -91,7 +91,7 @@ static inline void cw_putcs_aligned(struct vc_data *vc, struct fb_info *info,
 	u8 *src;
 
 	while (cnt--) {
-		src = par->fontbuffer + (scr_readw(s++) & charmask) * cellsize;
+		src = par->rotated.buf + (scr_readw(s++) & charmask) * cellsize;
 
 		if (attr) {
 			cw_update_attr(buf, src, attr, vc);
@@ -127,7 +127,7 @@ static void cw_putcs(struct vc_data *vc, struct fb_info *info,
 	u8 *dst, *buf = NULL;
 	u32 vxres = GETVXRES(par->p, info);
 
-	if (!par->fontbuffer)
+	if (!par->rotated.buf)
 		return;
 
 	image.fg_color = fg;
@@ -215,14 +215,14 @@ static void cw_cursor(struct vc_data *vc, struct fb_info *info, bool enable,
 	char *src;
 	u32 vxres = GETVXRES(par->p, info);
 
-	if (!par->fontbuffer)
+	if (!par->rotated.buf)
 		return;
 
 	cursor.set = 0;
 
  	c = scr_readw((u16 *) vc->vc_pos);
 	attribute = get_attribute(info, c);
-	src = par->fontbuffer + ((c & charmask) * (w * vc->vc_font.width));
+	src = par->rotated.buf + ((c & charmask) * (w * vc->vc_font.width));
 
 	if (par->cursor_state.image.data != src ||
 	    par->cursor_reset) {
diff --git a/drivers/video/fbdev/core/fbcon_rotate.c b/drivers/video/fbdev/core/fbcon_rotate.c
index 74206f5a6e98..6cdbc96eeca6 100644
--- a/drivers/video/fbdev/core/fbcon_rotate.c
+++ b/drivers/video/fbdev/core/fbcon_rotate.c
@@ -18,34 +18,35 @@
 int fbcon_rotate_font(struct fb_info *info, struct vc_data *vc)
 {
 	struct fbcon_par *par = info->fbcon_par;
-	unsigned char *fontbuffer;
+	unsigned char *buf;
 	int ret;
 
-	if (vc->vc_font.data == par->fontdata &&
-	    par->p->con_rotate == par->cur_rotate)
+	if (par->p->fontdata == par->rotated.fontdata && par->rotate == par->rotated.buf_rotate)
 		return 0;
 
-	par->fontdata = vc->vc_font.data;
-	par->cur_rotate = par->p->con_rotate;
+	par->rotated.fontdata = par->p->fontdata;
+	par->rotated.buf_rotate = par->rotate;
 
 	if (info->fbops->fb_sync)
 		info->fbops->fb_sync(info);
 
-	fontbuffer = font_data_rotate(par->p->fontdata, vc->vc_font.width,
-				      vc->vc_font.height, vc->vc_font.charcount,
-				      par->rotate, par->fontbuffer, &par->fd_size);
-	if (IS_ERR(fontbuffer)) {
-		ret = PTR_ERR(fontbuffer);
+	buf = font_data_rotate(par->rotated.fontdata, vc->vc_font.width,
+			       vc->vc_font.height, vc->vc_font.charcount,
+			       par->rotated.buf_rotate, par->rotated.buf,
+			       &par->rotated.bufsize);
+	if (IS_ERR(buf)) {
+		ret = PTR_ERR(buf);
 		goto err_kfree;
 	}
 
-	par->fontbuffer = fontbuffer;
+	par->rotated.buf = buf;
 
 	return 0;
 
 err_kfree:
-	kfree(par->fontbuffer);
-	par->fontbuffer = NULL; /* clear here to avoid output */
+	kfree(par->rotated.buf);
+	par->rotated.buf = NULL; /* clear here to avoid output */
+	par->rotated.bufsize = 0;
 
 	return ret;
 }
diff --git a/drivers/video/fbdev/core/fbcon_ud.c b/drivers/video/fbdev/core/fbcon_ud.c
index a1981fa4701a..eaf08999e249 100644
--- a/drivers/video/fbdev/core/fbcon_ud.c
+++ b/drivers/video/fbdev/core/fbcon_ud.c
@@ -92,7 +92,7 @@ static inline void ud_putcs_aligned(struct vc_data *vc, struct fb_info *info,
 	u8 *src;
 
 	while (cnt--) {
-		src = par->fontbuffer + (scr_readw(s--) & charmask) * cellsize;
+		src = par->rotated.buf + (scr_readw(s--) & charmask) * cellsize;
 
 		if (attr) {
 			ud_update_attr(buf, src, attr, vc);
@@ -127,7 +127,7 @@ static inline void ud_putcs_unaligned(struct vc_data *vc,
 	u8 *src;
 
 	while (cnt--) {
-		src = par->fontbuffer + (scr_readw(s--) & charmask) * cellsize;
+		src = par->rotated.buf + (scr_readw(s--) & charmask) * cellsize;
 
 		if (attr) {
 			ud_update_attr(buf, src, attr, vc);
@@ -164,7 +164,7 @@ static void ud_putcs(struct vc_data *vc, struct fb_info *info,
 	u32 vyres = GETVYRES(par->p, info);
 	u32 vxres = GETVXRES(par->p, info);
 
-	if (!par->fontbuffer)
+	if (!par->rotated.buf)
 		return;
 
 	image.fg_color = fg;
@@ -262,14 +262,14 @@ static void ud_cursor(struct vc_data *vc, struct fb_info *info, bool enable,
 	u32 vyres = GETVYRES(par->p, info);
 	u32 vxres = GETVXRES(par->p, info);
 
-	if (!par->fontbuffer)
+	if (!par->rotated.buf)
 		return;
 
 	cursor.set = 0;
 
  	c = scr_readw((u16 *) vc->vc_pos);
 	attribute = get_attribute(info, c);
-	src = par->fontbuffer + ((c & charmask) * (w * vc->vc_font.height));
+	src = par->rotated.buf + ((c & charmask) * (w * vc->vc_font.height));
 
 	if (par->cursor_state.image.data != src ||
 	    par->cursor_reset) {
-- 
2.53.0


^ permalink raw reply related	[flat|nested] 23+ messages in thread

* Re: [PATCH v2 05/10] lib/fonts: Implement glyph rotation
  2026-04-07  9:23 ` [PATCH v2 05/10] lib/fonts: Implement glyph rotation Thomas Zimmermann
@ 2026-04-07 15:48   ` Helge Deller
  2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
  1 sibling, 0 replies; 23+ messages in thread
From: Helge Deller @ 2026-04-07 15:48 UTC (permalink / raw)
  To: Thomas Zimmermann, gregkh, jirislaby, geert, simona, sam
  Cc: linux-fbdev, dri-devel, linux-kernel, linux-serial

Hi Thomas,

On 4/7/26 11:23, Thomas Zimmermann wrote:
> Move the glyph rotation helpers from fbcon to the font library. Wrap them
> behind clean interfaces. Also clear the output memory to zero. Previously,
> the implementation relied on the caller to do that.
> 
> Go through the fbcon code and callers of the glyph-rotation helpers. In
> addition to the font rotation, there's also the cursor code, which uses
> the rotation helpers.
> 
> The font-rotation relied on a single memset to zero for the whole font.
> This is now multiple memsets on each glyph. This will be sorted out when
> the font library also implements font rotation.
> 
> Building glyph rotation in the font library still depends on
> CONFIG_FRAMEBUFFER_CONSOLE_ROTATION=y. If we get more users of the code,
> we can still add a dedicated Kconfig symbol to the font library.
> 
> No changes have been made to the actual implementation of the rotate_*()
> and pattern_*() functions. These will be refactored as separate changes.
> 
> v2:
> - fix typos
> 
> Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
> ---
>   drivers/video/fbdev/core/fbcon_ccw.c    |   4 +-
>   drivers/video/fbdev/core/fbcon_cw.c     |   4 +-
>   drivers/video/fbdev/core/fbcon_rotate.c |  12 +-
>   drivers/video/fbdev/core/fbcon_rotate.h |  71 -----------
>   include/linux/font.h                    |   8 ++
>   lib/fonts/Makefile                      |   1 +
>   lib/fonts/font_rotate.c                 | 150 ++++++++++++++++++++++++

Patch is Ok.
But since you move/add the file lib/fonts/font_rotate.c,
it should be reflected in MAINTAINERS file.
Do you mind sending a follow-up patch which adds /lib/fonts/* to the
FRAMEBUFFER LAYER and FRAMEBUFFER CORE entries in the MAINTAINERS file?

Other than that, I've now added this series to the fbdev git tree.

Thanks!
Helge

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Claude review: fbcon,fonts: Refactor framebuffer console rotation
  2026-04-07  9:23 [PATCH v2 00/10] fbcon,fonts: Refactor framebuffer console rotation Thomas Zimmermann
                   ` (9 preceding siblings ...)
  2026-04-07  9:23 ` [PATCH v2 10/10] fbcon: Put font-rotation state into separate struct Thomas Zimmermann
@ 2026-04-12  3:55 ` Claude Code Review Bot
  10 siblings, 0 replies; 23+ messages in thread
From: Claude Code Review Bot @ 2026-04-12  3:55 UTC (permalink / raw)
  To: dri-devel-reviews

Overall Series Review

Subject: fbcon,fonts: Refactor framebuffer console rotation
Author: Thomas Zimmermann <tzimmermann@suse.de>
Patches: 12
Reviewed: 2026-04-12T13:55:01.507925

---

This is a well-structured 10-patch series from Thomas Zimmermann refactoring fbcon console rotation. It moves glyph and font rotation logic out of fbcon's private headers into the `lib/fonts/` library, creates clean public APIs, consolidates duplicated cursor mask code, and tidies up the rotation state in `fbcon_par`. The series begins with a genuine OOB fix (patch 1) and builds progressively through helper introduction, code movement, refactoring, and finally state reorganization.

The overall design direction is sound: rotation is a property of fonts/glyphs, not fbcon internals, and belongs in `lib/fonts/`. The series is nicely decomposed into reviewable increments.

However, **patch 7 introduces a correctness bug in the 180-degree rotation** for fonts whose width is not a multiple of 8 (i.e., most fonts). This needs to be fixed before merging.

The series depends on a prerequisite series (the `font_data_t` / `vc_font` work) which explains why the patches don't apply to the current drm-next tree.

---

---
Generated by Claude Code Patch Reviewer

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Claude review: fbcon: Avoid OOB font access if console rotation fails
  2026-04-07  9:23 ` [PATCH v2 01/10] fbcon: Avoid OOB font access if console rotation fails Thomas Zimmermann
@ 2026-04-12  3:55   ` Claude Code Review Bot
  0 siblings, 0 replies; 23+ messages in thread
From: Claude Code Review Bot @ 2026-04-12  3:55 UTC (permalink / raw)
  To: dri-devel-reviews

Patch Review

**Verdict: Good fix, correct for stable backport.**

The original code in `fbcon_rotate_font()` keeps the old (too-small) `fontbuffer` when reallocation for a larger rotated font fails. Subsequent character output can then overflow the buffer. The fix clears the buffer state *before* attempting allocation:

```c
+		kfree(par->fontbuffer);
+		par->fontbuffer = NULL;
+		par->fd_size = 0;
+
 		dst = kmalloc_array(len, d_cellsize, GFP_KERNEL);
```

This is correct. When `fontbuffer` is NULL, the rotated `putcs` implementations (`ccw_putcs`, `cw_putcs`, `ud_putcs`) check `if (!par->fontbuffer) return;` and bail out safely. The Fixes tag and Cc:stable are appropriate for a bug dating to v2.6.15.

No issues.

---

---
Generated by Claude Code Patch Reviewer

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Claude review: vt: Implement helpers for struct vc_font in source file
  2026-04-07  9:23 ` [PATCH v2 02/10] vt: Implement helpers for struct vc_font in source file Thomas Zimmermann
@ 2026-04-12  3:55   ` Claude Code Review Bot
  0 siblings, 0 replies; 23+ messages in thread
From: Claude Code Review Bot @ 2026-04-12  3:55 UTC (permalink / raw)
  To: dri-devel-reviews

Patch Review

**Verdict: Fine, straightforward.**

Moves `vc_font_pitch()` and `vc_font_size()` from inline in `console_struct.h` to out-of-line in `vt.c`. These are not hot-path functions, so the loss of inlining is irrelevant. Removes `#include <linux/math.h>` from the widely-included header, which is a minor positive.

Already has Acked-by from Greg KH. No issues.

---

---
Generated by Claude Code Patch Reviewer

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Claude review: lib/fonts: Provide helpers for calculating glyph pitch and size
  2026-04-07  9:23 ` [PATCH v2 03/10] lib/fonts: Provide helpers for calculating glyph pitch and size Thomas Zimmermann
@ 2026-04-12  3:55   ` Claude Code Review Bot
  0 siblings, 0 replies; 23+ messages in thread
From: Claude Code Review Bot @ 2026-04-12  3:55 UTC (permalink / raw)
  To: dri-devel-reviews

Patch Review

**Verdict: Good cleanup, one minor doc nit.**

Introduces `font_glyph_pitch(width)` and `font_glyph_size(width, vpitch)` as static inlines in `<linux/font.h>`, replacing open-coded `(width + 7) / 8` and `((width + 7) / 8) * height` throughout fbcon. The replacement is mechanical and correct throughout all callers (`fbcon_ccw.c`, `fbcon_cw.c`, `fbcon_ud.c`, `fbcon_rotate.c`, `fonts.c`, `vt.c`).

**Minor nit**: The kernel-doc for `font_glyph_pitch` says "if width is 14 bytes per scanline" -- this should say "14 **bits** per scanline", since the `width` parameter is in bits and the return value is in bytes.

---

---
Generated by Claude Code Patch Reviewer

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Claude review: lib/fonts: Clean up Makefile
  2026-04-07  9:23 ` [PATCH v2 04/10] lib/fonts: Clean up Makefile Thomas Zimmermann
@ 2026-04-12  3:55   ` Claude Code Review Bot
  0 siblings, 0 replies; 23+ messages in thread
From: Claude Code Review Bot @ 2026-04-12  3:55 UTC (permalink / raw)
  To: dri-devel-reviews

Patch Review

**Verdict: Fine.**

Switches from the older `font-objs` / `font-objs-$(CONFIG_...)` pattern to the modern `font-y` / `font-$(CONFIG_...)` Kbuild idiom. Sorts fonts alphabetically by Family-Size. Clean prep for the `font_rotate.o` addition in patch 5.

No issues.

---

---
Generated by Claude Code Patch Reviewer

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Claude review: lib/fonts: Implement glyph rotation
  2026-04-07  9:23 ` [PATCH v2 05/10] lib/fonts: Implement glyph rotation Thomas Zimmermann
  2026-04-07 15:48   ` Helge Deller
@ 2026-04-12  3:55   ` Claude Code Review Bot
  1 sibling, 0 replies; 23+ messages in thread
From: Claude Code Review Bot @ 2026-04-12  3:55 UTC (permalink / raw)
  To: dri-devel-reviews

Patch Review

**Verdict: Correct code movement.**

Moves `pattern_test_bit`, `pattern_set_bit`, `rotate_cw`, `rotate_ud`, `rotate_ccw` from `fbcon_rotate.h` (static inlines) into `lib/fonts/font_rotate.c`. Wraps them in public API functions (`font_glyph_rotate_90/180/270`) that add a `memset(out, 0, ...)` before calling the internal rotate. This is important since the `set_bit` functions only OR bits in.

The commit message correctly notes that this trades one bulk `memset` (old code zeroed the entire font buffer) for per-glyph `memset` calls, but this will be optimized again in patch 8. The `EXPORT_SYMBOL_GPL` is appropriate since the callers are in a different module.

The interface change placing `out` as the last parameter (matching kernel convention for output buffers) is a nice touch.

No issues with the logic -- this is a pure code movement with no behavioral changes to the rotation algorithms themselves.

---

---
Generated by Claude Code Patch Reviewer

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Claude review: lib/fonts: Refactor glyph-pattern helpers
  2026-04-07  9:23 ` [PATCH v2 06/10] lib/fonts: Refactor glyph-pattern helpers Thomas Zimmermann
@ 2026-04-12  3:55   ` Claude Code Review Bot
  0 siblings, 0 replies; 23+ messages in thread
From: Claude Code Review Bot @ 2026-04-12  3:55 UTC (permalink / raw)
  To: dri-devel-reviews

Patch Review

**Verdict: Clean refactor.**

Introduces `__font_glyph_pos()` to centralize the bit position calculation, plus `font_glyph_test_bit()` and `font_glyph_set_bit()`. The new helpers change signatures from `(x, y, pitch, pat)` to `(glyph, x, y, bit_pitch)` -- buffer-first, consistent with other `font_glyph_*` APIs.

The refactoring is purely mechanical. The rotation functions themselves are updated to call the new helpers but the bit manipulation logic is unchanged. Verified that the coordinate computations in all three rotation functions remain identical.

No issues.

---

---
Generated by Claude Code Patch Reviewer

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Claude review: lib/fonts: Refactor glyph-rotation helpers
  2026-04-07  9:23 ` [PATCH v2 07/10] lib/fonts: Refactor glyph-rotation helpers Thomas Zimmermann
@ 2026-04-12  3:55   ` Claude Code Review Bot
  0 siblings, 0 replies; 23+ messages in thread
From: Claude Code Review Bot @ 2026-04-12  3:55 UTC (permalink / raw)
  To: dri-devel-reviews

Patch Review

**Verdict: BUG in `__font_glyph_rotate_180` -- incorrect output coordinate for non-byte-aligned widths.**

This patch renames variables and drops `inline`, which is all fine. However, it introduces a bug in the 180-degree rotation.

In the **original** `rotate_ud()`, `width` was overwritten to `(width + 7) & ~7` (the bit pitch), and the output x-coordinate was computed as:

```c
width = (width + 7) & ~7;  // width is now bit_pitch
...
pattern_set_bit(width - (1 + j + shift), height - (1 + i), width, out);
```

Algebraically: `output_x = bit_pitch - 1 - j - shift`. Since `bit_pitch = width_orig + shift`, this simplifies to `output_x = width_orig - 1 - j`. The shift cancels out.

In the **new** `__font_glyph_rotate_180()`, the parameter `width` is no longer overwritten, but the formula still includes the shift:

```c
unsigned int bit_pitch = font_glyph_bit_pitch(width);
...
font_glyph_set_bit(out, width - (1 + x + shift), height - (1 + y), bit_pitch);
```

This computes `output_x = width - 1 - x - shift`, which is **wrong** -- it should be `width - 1 - x` (without the shift), or equivalently `bit_pitch - 1 - x - shift` (preserving the original formula with `bit_pitch`).

For any font width that isn't a multiple of 8 (e.g., 6, 7, 10, 12, 14...), the 180-degree rotation will place bits at incorrect positions. For example, with width=10, bit (0,0) maps to output position 3 instead of 9.

The 90-degree and 270-degree rotations are correct in this patch because they consistently use `out_bit_pitch` / `bit_pitch` (the rounded-up values) rather than the original width/height.

**Fix**: Change:
```c
font_glyph_set_bit(out, width - (1 + x + shift), height - (1 + y), bit_pitch);
```
to:
```c
font_glyph_set_bit(out, bit_pitch - (1 + x + shift), height - (1 + y), bit_pitch);
```

This wasn't caught in testing because the author tested with bochs+QEMU, which likely uses the default 8x16 font (width=8, shift=0).

---

---
Generated by Claude Code Patch Reviewer

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Claude review: lib/fonts: Implement font rotation
  2026-04-07  9:23 ` [PATCH v2 08/10] lib/fonts: Implement font rotation Thomas Zimmermann
@ 2026-04-12  3:55   ` Claude Code Review Bot
  0 siblings, 0 replies; 23+ messages in thread
From: Claude Code Review Bot @ 2026-04-12  3:55 UTC (permalink / raw)
  To: dri-devel-reviews

Patch Review

**Verdict: Good design, minor style nit.**

Introduces `font_data_rotate()` which takes a full font's glyph data and produces a rotated copy. Key design decisions are reasonable:

- `steps % 4` for normalization
- Buffer reuse via `buf`/`bufsize` parameters (avoids reallocation on repeated rotation)
- `check_mul_overflow` for safety
- Returns `ERR_PTR` on failure
- Restores the bulk `memset` optimization (one memset for the entire output buffer, then per-glyph rotation without zeroing)

The `krealloc`-like reuse semantics are well-documented.

**Minor**: The switch for `d_cellsize` has no `default` case. While all values 0-3 are covered after `steps %= 4`, some compilers may warn about potentially uninitialized `d_cellsize`. A `default:` matching case 0/2 would silence this.

The caller-side refactoring in `fbcon_rotate_font` is clean. The error path correctly handles the case where `font_data_rotate` returns ERR_PTR without freeing the old buffer (the caller frees it).

The `fd_size` type change from `u32` to `size_t` is appropriate.

---

---
Generated by Claude Code Patch Reviewer

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Claude review: fbcon: Fill cursor mask in helper function
  2026-04-07  9:23 ` [PATCH v2 09/10] fbcon: Fill cursor mask in helper function Thomas Zimmermann
@ 2026-04-12  3:55   ` Claude Code Review Bot
  0 siblings, 0 replies; 23+ messages in thread
From: Claude Code Review Bot @ 2026-04-12  3:55 UTC (permalink / raw)
  To: dri-devel-reviews

Patch Review

**Verdict: Correct consolidation, minor style nit.**

Consolidates four copies of cursor mask creation into `fbcon_fill_cursor_mask()`. The function creates a standard (non-rotated) mask, and the rotation-specific cursor functions then rotate it. This is a clean decomposition.

Verified that the behavioral change in `ud_cursor` (previously built the mask in reversed order, now builds normal + rotates 180) produces the same visual result. For full-width masks (all bits set or clear per row), the horizontal flip in 180-degree rotation is a no-op, so only the vertical flip matters, which correctly inverts the row order.

**Minor nit**: The `par` parameter to `fbcon_fill_cursor_mask()` is unused in the function body. The function only uses `vc` and `mask`. Consider removing the `par` parameter, or if it's intended for future use, document why it's there.

The `static const unsigned int pattern = 0xffffffff` with casts to `unsigned char` is a bit roundabout. A simple `0xff` / `0x00` would be clearer, but this is harmless and was apparently chosen to fix a sparse warning.

---

---
Generated by Claude Code Patch Reviewer

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Claude review: fbcon: Put font-rotation state into separate struct
  2026-04-07  9:23 ` [PATCH v2 10/10] fbcon: Put font-rotation state into separate struct Thomas Zimmermann
@ 2026-04-12  3:55   ` Claude Code Review Bot
  0 siblings, 0 replies; 23+ messages in thread
From: Claude Code Review Bot @ 2026-04-12  3:55 UTC (permalink / raw)
  To: dri-devel-reviews

Patch Review

**Verdict: Good organization.**

Moves all rotation-related state into a nested `struct rotated` inside `fbcon_par`, guarded by `CONFIG_FRAMEBUFFER_CONSOLE_ROTATION`. This cleanly separates rotation concerns from core fbcon state and avoids allocating unused fields when rotation is disabled.

The field rename `fontdata` to `rotated.fontdata` (now typed as `font_data_t *`) and comparison change from `vc->vc_font.data` to `par->p->fontdata` is consistent with the prerequisite series's introduction of `font_data_t`. The rotation comparison change from `par->p->con_rotate` to `par->rotate` is semantically correct (checking the par's own rotation value against what was cached).

The `#ifdef` guards around `kfree(par->rotated.buf)` in `fbcon_release` and initialization in `fbcon_startup` are necessary and correct.

No issues.

---

**Summary**: The series is well-crafted overall but **patch 7 has a bug in `__font_glyph_rotate_180`** that will corrupt 180-degree rotation for fonts with non-byte-aligned widths. This needs to be fixed. All other patches look correct.

---
Generated by Claude Code Patch Reviewer

^ permalink raw reply	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2026-04-12  3:55 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-04-07  9:23 [PATCH v2 00/10] fbcon,fonts: Refactor framebuffer console rotation Thomas Zimmermann
2026-04-07  9:23 ` [PATCH v2 01/10] fbcon: Avoid OOB font access if console rotation fails Thomas Zimmermann
2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
2026-04-07  9:23 ` [PATCH v2 02/10] vt: Implement helpers for struct vc_font in source file Thomas Zimmermann
2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
2026-04-07  9:23 ` [PATCH v2 03/10] lib/fonts: Provide helpers for calculating glyph pitch and size Thomas Zimmermann
2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
2026-04-07  9:23 ` [PATCH v2 04/10] lib/fonts: Clean up Makefile Thomas Zimmermann
2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
2026-04-07  9:23 ` [PATCH v2 05/10] lib/fonts: Implement glyph rotation Thomas Zimmermann
2026-04-07 15:48   ` Helge Deller
2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
2026-04-07  9:23 ` [PATCH v2 06/10] lib/fonts: Refactor glyph-pattern helpers Thomas Zimmermann
2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
2026-04-07  9:23 ` [PATCH v2 07/10] lib/fonts: Refactor glyph-rotation helpers Thomas Zimmermann
2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
2026-04-07  9:23 ` [PATCH v2 08/10] lib/fonts: Implement font rotation Thomas Zimmermann
2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
2026-04-07  9:23 ` [PATCH v2 09/10] fbcon: Fill cursor mask in helper function Thomas Zimmermann
2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
2026-04-07  9:23 ` [PATCH v2 10/10] fbcon: Put font-rotation state into separate struct Thomas Zimmermann
2026-04-12  3:55   ` Claude review: " Claude Code Review Bot
2026-04-12  3:55 ` Claude review: fbcon,fonts: Refactor framebuffer console rotation Claude Code Review Bot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox