错误 #5583
版本R1.1.0_T01,DD口,ue1和ue2多次脱网及再在网后,ue1不能发出hello广播
0%
描述
高概率发生。
文件
历史记录
由 李 玮璇 更新于 11 天 之前
从日志看,调整天线多次释放建立流程后,出问题时nc没有日志打印,threadL3无法正常进入业务,像是while已经无法触动下一轮,但是threadL3的核却一直在小号cpu,dpdk的timer管理陷入某种异常无法循环业务导致nc无法发送hello
此情况之前有过偶现
[Switching to thread 11 (Thread 0x7f73ffd900 (LWP 980))]
#0 0x0000000000dae7c0 in __rte_timer_reset ()
(gdb) bt
#0 0x0000000000dae7c0 in __rte_timer_reset ()
#1 0x0000000000daf464 in rte_timer_manage ()
#2 0x00000000008707e4 in wnTimerManage () at fwk/ngTimer/csrc/wn5gNrPsTimerApi.c:82
#3 0x0000000000898ba0 in threadL3 (ueCb=0x384a5000) at src/l3/rrc/csrc/yzL3Mani.c:597
#4 0x0000000000dbd0d8 in eal_thread_loop ()
#5 0x0000007f959da7e4 in start_thread (arg=0x7fd6c2ce4f) at pthread_create.c:486
#6 0x0000007f9586070c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78
(gdb) quit
由 李 玮璇 更新于 4 天 之前
【问题现象一】threadL3卡在__rte_timer_reset
(gdb) p *((struct rte_timer *)0x1006d788)
$1 = {expire = 14744347765, sl_next = {0x1006d788, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, status = {{state = 3, owner = 2}, u32 = 131075}, period = 0,
f = 0x969c80 <wnSacAuthTimerMsgSend>, arg = 0x187c69480}
- Children Self Command Shared Object Symbol
- ........ ........ ........ ................ ......................
#
60.27% 60.27% threadL3 psarmapp [.] 0x00000000009af880
39.73% 39.73% threadL3 psarmapp [.] 0x00000000009af88c
Line 74 of "src/l2/mac/csrc/cm_mblk.c" starts at address 0x9af884 <cmAllocEvnt+84>
and ends at 0x9af89c <cmAllocEvnt+108>.
(gdb) info line *0x00000000009af880
Line 72 of "src/l2/mac/csrc/cm_mblk.c" starts at address 0x9af880 <cmAllocEvnt+80>
and ends at 0x9af884 <cmAllocEvnt+84>.
【问题现象二】hello定时器丢失,但是线程不卡死,sac定时器还能处理
基本确认在反复重建下,有些场景会导致dpdk定时器异常。目前确认触发业务相似操作中。。。只发现sac定时器快到期时触发鉴权开始重置定时器后容易出问题,但是因不涉及跨核同步操作所以无法确认根因;
0731在西安分享后,决定从以下方向定位:
1、在线程循环打印中把hello、sac的回调函数地址打印出来
2、问题发生时业务相同性
3、操作隔离,定时器隔离
4、rte_timer_dump_stats(stderr);挪在外部周期打印,关注定时器数量