星期五, 12月 25, 2020

This has been noted in various places around the web but in practice what I did seems to be a combination of various writings so I have documented my own experiences here.

Background

I recently acquired a (yet another) USB flash drive, this a 16 GB “Dolphin” brand. The actual device reports as “048d:1165 Integrated Technology Express, Inc.” when interrogated using lsusb. I am using it to transfer transcoded Kaffeine PVR recordings from my PC to the set top box in the lounge for more comfortable watching.

On first use, however, it took what seemed like forever to transfer a 250MB AVI file, over USB2, and looking at the GKrellM chart the write data rate appeared to be a very poor 350 kB/sec. So it seemed yet again, I needed to optimise a USB disk before it was adequate for use.

In theory, to simplify things to one sentence, flash disk (and in particular, modern SSD) should be faster than spinning disks, as access is a true physical random access operation, without having to wait for the heads to be in the right spot. However this is invalidated due the blocky nature of flash disk writes. The actual reason for the poor write speed is that the default partition starts at the 63rd sector (byte 32256) on the disk, and USB flash drives, SD cards, etc. are designed to write data in chunks of say 128kB at a time. Even if you only write one sector, the entire 128kB (or 256 sectors) must be (re-read first and) written. So when a partition is not aligned on a 128kB boundary, more writes than otherwise necessary are required, slowing performance. USB flash drives generally employ FAT32 so they are usable on the widest variety of devices (including set top boxes) and the general experience of FAT32 is that write performance is severely affected if the partition alignment does not match the flash write size, for both the partition and the FAT master table itself.

Procedure

The procedure I follow for doing fixing misaligned flash drives is:

1. Find a Linux computer, or reboot using a live Linux distribution such as [SysRescueCD](http://www.sysresccd.org/SystemRescueCd_Homepage)

2. Destroy the existing partition.

3. Recreate a single partition, ensuring it starts at the 256th sector (byte 131072, or 128kB)

4. Format the partition to FAT32. with the following non default options:

  * override the default sectors per cluster to ensure clusters are aligned.  This comes at some expense of apparent usable space, but the performance gain for writing large files such as video files is more than worth it.
  * Adjust the "reserved" sectors so that the FAT table itself is aligned to 128kB.

Detailed Steps

The following command sequence will accomplish this under Linux. This assumes your drive is at /dev/sdd, this will vary depending on what other disks you have.

1. Run GNU fdisk with units in sector mode not cylinder mode.  Then print the existing partition table (enter _p_ when prompted.  Below you can see the start sector of the existing partition is at sector 63.  Note this is also a primary partition.  This is typical of USB flash disks you might purchase at the local supermarket... ```bash fdisk -u /dev/sdd``` ``` GNU Fdisk 1.2.4 Copyright (C) 1998 - 2006 Free Software Foundation, Inc. This program is free software, covered by the GNU General Public License.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

Using /dev/sdd Command (m for help): p

Disk /dev/sdd: 16 GB, 16162675200 bytes 255 heads, 63 sectors/track, 1965 cylinders, total 31567725 sectors Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System /dev/sdd1 63 31570943 15791863 c FAT32 LBA Warning: Partition 1 does not end on cylinder boundary.
Command (m for help):




    2. Delete the partition:

Command (m for help): d Partition number (1-1): 1
Command (m for help): p

Disk /dev/sdd: 16 GB, 16162675200 bytes 255 heads, 63 sectors/track, 1965 cylinders, total 31567725 sectors Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System Command (m for help):

    3. Recreate the partition, aligned at sector 256 (131072 bytes), and set the type back to FAT32 LBA (in this case matching what previously existed) (type 'c', or 0x0c, i.e. FAT32 LBA).  Use of FAT32 LBA allows use to start the filesystem on an arbitrary sector bearing no relationship to legacy cylinders, etc.  The final sector depends on the disk size.

Command (m for help): n
Partition type
e extended p primary partition (1-4) p First sector (default 63s): 256s
Last sector or +size or +sizeMB or +sizeKB (default 31567724s):
Command (m for help): t
Partition number (1-1): 1
Hex code (type L to list codes): c
Changed type of partition 1 to c (FAT32 LBA) Command (m for help): p

Disk /dev/sdd: 16 GB, 16162675200 bytes 255 heads, 63 sectors/track, 1965 cylinders, total 31567725 sectors Units = sectors of 1 * 512 = 512 bytes

Device Boot Start End Blocks Id System /dev/sdd1 256 31567724 15783831 c FAT32 LBA Command (m for help):




    4. Save changes:

Command (m for help): w
Information: Don’t forget to update /etc/fstab, if necessary.

Writing all changes to /dev/sdd.

    5. Format the partition, setting the number of reserved sectors so that the FAT table remains aligned at a 128kB boundary.  Assuming sectors per cluster, s=128 (65536 bytes), and our partition length of 31567469 sectors, we want the first fat to start at the 256th sector within the partition (which is OK as the partition itself is aligned.)  For some sizes of flash disk, this can be an iterative process, but generally setting the number of reserved sectors to 256 will achieve what we want.
```sh
mkfs.vfat -v -F 32 -n label -s 128 -R 256 /dev/sdd1

mkfs.vfat 3.0.9 (31 Jan 2010)
/dev/sdi1 has 255 heads and 63 sectors per track,
logical sector size is 512,
using 0xf8 media descriptor, with 31567468 sectors;
file system has 2 32-bit FATs and 128 sectors per cluster.
FAT size is 2048 sectors, and provides 246586 clusters.
There are 256 reserved sectors.
Volume ID is 3bd81e55, volume label fatflash   

This is the most important step - verify that the chosen number of reserved sectors has resulted in an aligned FAT table and aligned data area. ```sh fsck.vfat /dev/sdd1 ``` ``` fsck from util-linux-ng 2.17.2 dosfsck 3.0.9 (31 Jan 2010) dosfsck 3.0.9, 31 Jan 2010, FAT32, LFN Checking we can access the last sector of the filesystem Boot sector contents: System ID "mkdosfs" Media byte 0xf8 (hard disk)
bytes per logical sector
bytes per cluster
reserved sectors First FAT starts at byte 131072 (sector 256)
FATs, 32 bit entries    1048576 bytes per FAT (= 2048 sectors) Root directory start at cluster 2 (arbitrary size) Data area starts at byte 2228224 (sector 4352)
246586 data clusters (16160260096 bytes) 63 sectors/track, 255 heads
hidden sectors   31567468 sectors total Checking for unused clusters. Checking free cluster summary. /dev/sdd1: 1 files, 1/246586 clusters ``` The important figure here, is the data area sector - it must be an integer multiple of 256, and 256 x 17 == 4362 in this example.



Test the result.  I copied a 256 MB file onto the drive, and GKrellM is now reporting ~2.5MB/sec.  More importantly, it finished in approx. one eighth of the time compared to before reformatting.

The improved write performance should be just as noticeable from Windows.

posted in disks, linux, performance, usb

星期四, 11月 12, 2020

docker for ubuntu 16

https://stackoverflow.com/questions/30379381/docker-command-not-found-even-though-installed-with-apt-get

For Ubuntu 14.04/16.04/16.10/17.04:

sudo add-apt-repository "deb [arch=amd64] \
     https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

For Ubuntu 17.10:

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu zesty stable"

Add Docker’s official GPG key:

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Then install

$ sudo apt-get update && sudo apt-get -y install docker-ce

janus websocket

janus

https://janus.conf.meetecho.com/index.html

https://stackoverflow.com/questions/61672575/rtsp-to-webrtc-live-video-streaming

https://groups.google.com/forum/#!topic/meetecho-janus/goc1d2K0IG0

RTSPtoWebRTC golang

https://github.com/deepch/RTSPtoWebRTC

Virtual Webcam Driver

星期一, 11月 09, 2020

3+1 H.264 基礎

視訊壓縮圖像類型

https://zh.wikipedia.org/wiki/%E8%A6%96%E8%A8%8A%E5%A3%93%E7%B8%AE%E5%9C%96%E5%83%8F%E9%A1%9E%E5%9E%8B

3+1 H.264 基礎

http://chpsocket.blogspot.com/2017/10/31-h264.html

ntpclient

https://github.com/Telconet/ntpclient

VT100終端中printf顏色

http://chunchaichang.blogspot.com/2011/07/printf.html

VT100終端中printf顏色

echo也能加顏色，只是必須要用echo -e ""（後面同printf）

星期二, 11月 03, 2020

在後台運行命令與前景終端接入 backgroup system stdin

https://unix.stackexchange.com/questions/507786/run-command-in-background-with-foreground-terminal-access

星期五, 10月 23, 2020

struct enum

2、填充結構
#define FILL(a) {a, #a}

enum IDD{OPEN, CLOSE};
typedef struct MSG{
IDD id;
const char * msg;
}MSG;

MSG _msg[] = {FILL(OPEN), FILL(CLOSE)};
相當於：
MSG _msg[] = {{OPEN, "OPEN"},
{CLOSE, "CLOSE"}};

http://stenlyho.blogspot.com/2007/04/c_04.html

星期四, 10月 22, 2020

utf-8 en/decode ascii 88591 88599

https://github.com/haipome/utf8

https://en.wikipedia.org/wiki/ISO/IEC_8859-9

星期三, 10月 07, 2020

Wi-Fi SDIO blocks the system's Supend -to-RAM

星期二, 10月 06, 2020

AVS Device SDK 架構

在 Raspberry Pi 上使用 AVS Device SDK 實現智慧音箱

星期二, 9月 22, 2020

ProtobufCMessage

struct ProtobufCMessage {

const ProtobufCMessageDescriptor */;

-----------------

struct ProtobufCMessageDescriptor {

ProtobufCMessageInit message_init;

const ProtobufCFieldDescriptor *fields;

uint32_t magic;

const char *name;

----------------------

struct ProtobufCFieldDescriptor

--------------------------------------------

const ProtobufCMessageDescriptor vpi__descriptor =

{

PROTOBUF_C__MESSAGE_DESCRIPTOR_MAGIC,

"vpi",

"Vpi",

"",

sizeof(Vpi),

vpi__field_descriptors,

vpi__field_indices_by_name,

1, vpi__number_ranges,

(ProtobufCMessageInit) vpi__init,

NULL,NULL,NULL /* reserved[123] */

};

--------------------------------------------

void vpi__init (Vpi *message){

static const Vpi init_value = VPI__INIT;

*message = init_value;

}

--------------------------------------------

#define ETHERNET__INIT \

{ PROTOBUF_C_MESSAGE_INIT (&ethernet__descriptor) \

, 0, 0, (char *)protobuf_c_empty_string, (char *)protobuf_c_empty_string, (char *)protobuf_c_empty_string, (char *)protobuf_c_empty_string }

#define PROTOBUF_C_MESSAGE_INIT(descriptor) { descriptor, 0, NULL }

--------------------------------------------

protobuf_c_message_init(const ProtobufCMessageDescriptor * descriptor, void *message){

descriptor->message_init((ProtobufCMessage *) (message));

}

星期二, 9月 01, 2020

Yocto

Yocto基本概念及介紹

星期四, 8月 27, 2020

光標的恢復隐藏

linux shell光标的恢复与隐藏

echo -e "\033[?25l" 隐藏光标

echo -e "\033[?25h" 显示光标

其实\033(八进制,相当于10进制的27)就是<ctrl+v><ESC>所输入字符的值.

echo -ne "\33[32m" 可以将字符的显示颜色改为绿色

echo -ne "\33[3;1H" 可以将光标移到第3行第1列处

具体的摘抄一些如下：

\33[0m 关闭所有属性

\33[1m 设置高亮度

\33[4m 下划线

\33[5m 闪烁

\33[7m 反显

\33[8m 消隐

\33[30m -- \33[37m 设置前景色

\33[40m -- \33[47m 设置背景色

\33[nA 光标上移n行

\33[nB 光标下移n行

\33[nC 光标右移n行

\33[nD 光标左移n行

\33[y;xH设置光标位置

\33[2J 清屏

\33[K 清除从光标到行尾的内容

\33[s 保存光标位置

\33[u 恢复光标位置

\33[?25l 隐藏光标

\33[?25h 显示光标

字背景颜色范围:40----49

40:黑

41:深红

42:绿

43:黄色

44:蓝色

45:紫色

46:深绿

47:白色

字颜色:30-----------39

30:黑

31:红

32:绿

33:黄

34:蓝色

35:紫色

36:深绿

37:白色

不要忘了-e 和"",否则是起不了作用的.

二、使用linux c的printf函数实现

printf("\033[?25h");　　//显示光标

printf("\033[?25l");　　//隐藏光标

星期三, 8月 26, 2020

WSL2 win10 ubuntu

Windows 10 上適用於 Linux 的 Windows 子系統安裝指南 https://docs.microsoft.com/zh-tw/windows/wsl/install-win10 Window10 建置Ubuntu(WSL2)與GUI桌面配置筆記 https://medium.com/@s123600g/window10-%E5%BB%BA%E7%BD%AEubuntu-wsl2-%E8%88%87gui%E6%A1%8C%E9%9D%A2%E9%85%8D%E7%BD%AE%E7%AD%86%E8%A8%98-58796915ed4d ssh启动错误：no hostkeys available— exiting https://wangxianggit.github.io/sshd%20no%20hostkeys%20available/ Edit the /etc/ssh/sshd_config file. Change PasswordAuthentication and ChallengeResponseAuthentication to yes. sudo service ssh restart

星期二, 8月 25, 2020

GATT protocol

Performance Evaluation of Bluetooth Low Energy:A Systematic Review

The GATT is built over ATT protocol, which defines 2 terms

藍芽核心技術概述（一）:藍芽概述

Intel® Edison Bluetooth* Guide

GATT Git

BluezGATTService
bluez-gatt/btgatt-server.c
BLE in Linux One : Bluez GATT Server Concept

article

RPi Bluetooth LE

python/go gatt server example

蓝牙工具hcitool和gatttool的使用

L2CAP

RFCOMM

Bluetooth RFCOMM介绍

Discuss

linux - 藍牙低能耗：Android Gatt客戶端連接到Linux伺服器

我已經從Broadcom獲得關於這個 Bug的信息，我不認為它在它們的部分是 Bug 。如果一個沒有在廣告( 位 2 ) 中特別設置'不支持 br/edr'位，Broadcom棧將使用 br/edr和 BTLE 。將領先的廣告包從 020102改為 020106解決了問題。

訂閱：文章 (Atom)

星期五, 12月 25, 2020

星期二, 12月 22, 2020

星期五, 12月 04, 2020

星期五, 11月 13, 2020

9. Wowza Streaming Engine: RTSP Streaming

Background

Procedure

Detailed Steps

星期四, 11月 12, 2020

星期一, 11月 09, 2020

視訊壓縮圖像類型

3+1 H.264 基礎

星期二, 11月 03, 2020

星期五, 10月 23, 2020

星期四, 10月 22, 2020

星期三, 10月 07, 2020

星期二, 10月 06, 2020

星期二, 9月 22, 2020

星期二, 9月 01, 2020

星期四, 8月 27, 2020

星期三, 8月 26, 2020

星期二, 8月 25, 2020

ADS

Blogroll

標籤

網誌存檔

Archives

Human, Computing & Accessibility

Last Comments

大鏢客 的分享項目

關於我自己

Counter

Subscribe

大鏢客的分享項目